SANS Webcast: Running a Better Red Team Through Understanding ICS SCADA Adversary Tactics

hello everyone and welcome to today's sands webcast running a better red team through understanding ICS SCADA adversary tactics my name is Carol auth of the SANS Institute and I will be moderating today's webcast today's featured speaker is Robert Emily certified sands instructor and course author if during the webcast you have any questions for Robert please enter them into the questions window located on the go to webinar interface we will be answering them during the Q&A session at the end of the presentation and with that I'd like to hand the webcast over to Robert all right thank you so much well I appreciate you all tuning in today I was really fortunate to be asked to do this by Edna's folks and obviously is is it's goes queuing up in his team for hackfest if you haven't been to hack best before it's certainly worth checking out I'm always sort of jealous i'm over on the the ICS and the forensic side and we throw some pretty sweet conferences and summits and a lot of fun stuff but every now and then it pulls out something enact best like renting out the entirety of the air and space museum things like that that are just tons of fun so yeah so sort of kicking off the idea of doing hackfest this year in this summit I wanted to give a presentation since I can't attend on looking at ICS attacks and looking at the Espionage efforts against industrial control systems rice yes and the type of infrastructure that runs our power grids or water utilities or oil pipelines look at sort of what adversaries are doing there to help you in terms of running a better red team now I'm not going to go through you know here and here is Kali Linux here is different tools you can use I'm not going to look at you know how do you structure red team I want to take this next 55 minutes this next 45 minutes open up the questions and really focus on what the adversary has been doing and i'll get to that here in a moment just for a quick introduction for myself I'll note that I am NOT the other rod l-e-e-et's and so is going to start this off and including my a picture here because there are two robley as it stands both of us started off in the air force both of us where the intelligence community both of us were Denver Broncos fans all those backgrounds and forensics it gets a little silly so big rob bromley number one the the deeper curriculum lead here at sams I mostly work on the ICS side although I do have the cyber threat intelligence course on the forensic side as well I also run a company called Drago's or written where we do some fun stuff but I'm not going to pitch you on it what I want to focus on is my background in looking at hunting for ICS threats in ICS environments and bring some of that defender perspective on really what some some red team focuses could help for but also to bring in again somewhat offensive perspective I really do believe sort of EDS motto with a hackfest that offense does inform defense there the last year of my career the last two years my career in the military after my time in intelligence community I was benefited to be on an offensive team wasn't end testing wasn't a red team but an actual military offense team going after locations and seeing that offensive and defensive perspective was was very beneficial now I don't want anybody to be hacking infrastructure around the world and going on the offense but that red team perspective is very very beneficial for not only testing out our infrastructure but for testing out our defensive teams as well so a little bit of outline for we're going to cover today I want to start off not spend too much time but on two quick models I'm going to talk about the sliding scale cyber security and the ICS separate kill chain specifically to give a context to what I'm going to be briefing in the adversary tactics section that will be the heart of our presentation today and then I'll end up leaving you with a slide on recommendations for the red team's based off of these adversary tactics but I'll be sprinkling in those throughout as well there have only been a handful of public case studies on different ICS attacks you know you had the big one in Iran with Stuxnet but then the other two pieces of ICS tailored malware out there we're have X in black energy to and we'll talk about black energy to as well as black energy three but the two attacks that we saw in the community outside of Iran was the German still works attack in 2014 and the Ukraine attack and 2015 so I chose the two out of the three pieces of malware have X and black energy three and then the actual attack from Ukrainian which I was fortunate to work as an investigator i'm looking at writing the report and piecing things together is from from a lot of other good work in the community and then bring out sort of that perspective so that's what we'll focus on today so when we look at the models the reason i like models as they allow us an opportunity to sort of abstract or sell from data sets and pull out information so what i'll note here with this sliding scale of cyber security is i put this together and i wrote a paper on it that you can reference as well it's a sans paper because i saw a lot of folks talking about security and saying oh well i'm going to do something in invest in security or I'm I'm a cyber security person what does that mean you know what what exactly are you doing what area are you focusing on what direction race or driving organization and what I came to is really these five categories of potential investment on the left-hand side of the scale you have architecture that's the building out the network with security in mind the architect and yet networking maintaining and patching it the supply chain aspect you know the policies are rounded etc this step up from that as the passive defenses you're putting your firewalls your intrusion prevention systems your antivirus your endpoint agents you know your whatever next generation solution is on the market today you're putting that there that active defense component is when analysts are monitoring for responding to and learning format a series of their environment it's the human component its threat hunters it's instant responders its network security monitoring analysts security operations folks people who are focused on the threats and trying to find them in the environment that intelligence aspect is is generally around the idea of collecting data exploiting it and producing intelligence but it can also be counterintelligence and sort of using intelligence and a variety of different ways that will talk about then lastly is that offensive component which is those legal countermeasures you know when when Sony gets compromised and if the United States government comes out and says it's North Korea and wants to do something against North Korea you know there are certain legal countermeasures in the international community if they have technically a hack back now it's got to be for Security's sake but I want to note two different things here number one that hack back stuff is still offense see a lot of talk on this in terms of what we can do some cool things and maybe you know be like touch now it's so offense but the other component that I want to bring out with the scale in general is that all that value towards security is trying to push things to the left hand side if an organization does their security investments correctly they'll find that the return on investment for offense is is ridiculous compared to actually tuning their firewalls updating their systems empowering their defenders training the folks so it really when I put a lot of this together it was to get us off this idea of offense and push us back to trying to do everything from the cyber hygiene type basics to building out the security teams and environments now for a red team discussion this work is a little interesting I don't want to counter anybody else's opinions from sort of the sams crowd the pen test group here is is most certainly some world experts that being said from my perspective when I think of a pen test I think of validating architecture pen test is in that organ extra category it's seeing if the network is configured like it's supposed to be configured if the patch it is effective is their vulnerability and can i exploit it that's an architecture aspect and it's very very very useful from a red team perspective that's in the intelligence category of take counterintelligence action you are emulating a threat you were saying here is how the Ukraine power grid attack happened let me emulate that threat against your environment and see how you would respond see how you do against this style of attack so that's sort of the the way that I draw the lines here so when we're talking today and I'm showing these different adversary tactics it's not from that is this exploitable architecture perspective it's from understanding and emulating the threats that we've seen in the community and threats being the humans and the capability is an intent and opportunity to have not just the malware right so the other thing to structure our discussion is this idea of the ICS cyber kill chain there is almost no chance that you've been in this industry and haven't heard of the kill chain and the kill chain is great wonderful for a number of things sometimes some people get sort of phased out with it and maybe a little frustrated but the reason I like it is the ability to structure and visually show through that a high level what's going on and to be able to put data into different little pockets right so as I have a pattern of delivery how effective is that and what solutions am I putting in place to counter it as I have patterns and multiple samples of c2 command control what am i doing to counter that and working now on the ICS side though it's very similar right on the stage one so in the stage one it's almost identical to the Lockheed Martin kill chain the only difference is cyclic targeting there instead of just weaponization you can do either or both so if I have a UH natha cated unauthenticated VPN directly connecting me into a skate environment that actually has windows 10 running with powershell I don't need to really weaponize the VPN I'm just logging in and using the PowerShell environment against you have targeted your environment delivered exploited the environment even using native capabilities maybe just modified instead of installing something if it install malware maybe just modify you know how PowerShell is working in that environment so those are the two tailoring that I made here but the really hurt and soul of it is the fact that industrial control system environments are very different than IT networks we have physical components we have networking gear and infrastructure that's a little bit abnormal protocols that are different and there's no commonality between all of the ICS sites so if you want it to target one substation for electrical power grid in one portion of the country it would be a very different way that you had to understand the ICS and the industrial you know the control systems there than a different substation in a different part of the country different vendors different configuration different ways of being integrated so to actually do that if we're thinking of causing actual damage outages physical damage maybe even just logical in terms of causing you know power to go down water to stop oil to stop flowing etc that's that stage 2 so all the stage 1 stuff is just getting you to the point where you understand the ICS well enough to impact it almost more engineering than cyber stuff so in that stage 2 that's where you have to develop some sort of understanding to do the attack it could be developing your knowledge against the ice TS it could be developing a specific capability for the ICS and but at some point you'll have to test it since these environments are so different and because there are things that we can't plan for sometimes there is a real requirement to test out knowledge and capabilities especially for those large-scale attacks and things that we are more concerned about then there's that aspect of delivery installation modification and attack what I want to note here and I'm going to flip back and forth rather quickly is in this stage one to stage two style of effect what this really means is that to impact industrial control systems the attacker has to know more and has to do more to have a controlled effect can you get into an environment and cause some disruption does randomly sure if we're talking about attacks especially those high confidence scenarios that were concerned about things that can cause real issues that aren't going to get stopped by a safety system or good engineering there's this aspect of walking through and knowing more which in my mind makes ICS environments some of the most defensible on the planet which means red teaming to come in and figure out where the gaps are and help illuminate which steps were not really taken care of or countering very well is extremely helpful so let's jump in these adversary tactics and out lie them against that killed same model to talk about where you might take it of red teaming to test out the architecture pass defenses and active defenses of your defenders so have X a little bit of back grab a have X it was your sort of run-of-the-mill remote access Trojan what I liked about it most and and it's sort of harsh to say that I liked an adversary capability what I liked about him most was that it wasn't using zero days and all sorts of fancy things a lot of folks and they think of the advanced actors and the well-funded teams they think in terms of sophistication I will tell you having been on the government offensive teams there's no bonus points for looking good in the kaspersky report right the idea is do as little as possible where I do as little as you have to to achieve operation efficiency you want to be able to fly under the radar you want to be able to achieve success but if you look really really sophisticated at time is that's an operational risk because then folks think it's really creative and kool kool in the specialized teams maybe spend more time on it but if you can get in with a basic methodology you might fly under the radar even if you do get caught and that's basically what have x did it operated in from 2013-2014 and there were thousands of industrial control system sites around the world that were compromised effectively they used three different styles of intrusion there was the normal run-of-the-mill spearfishing the water hole attack which all that really was as they targeted ICS bender websites and as ICS companies came and looked at the water of the webpage they were compromised with well known exploits it was at some points copy and pasted metasploit modules not very advanced at all now the one that was more interesting with trojan eyes dice es software installers the reason this is interesting is an industrial control system facilities we often have different ways of segmenting out our environments and sometimes people don't do this but in some facilities in a lot of facilities we try for what we call the purdue model and in this environment we try to have a difference between the business network a difference between the plant network the tea see and actually we start getting down into the ICS into control areas and my point in saying this is if your spear fishing you're going to land in the business networks if you're doing a water hole attack you might land in the DMZ at best but if you actually do the Trojan eyes ICS software installers you can actually get into the industrial control networks themselves and so we saw this evolution in the adversary campaign almost as if they were realizing that as well maybe they were but we can't assume intent based off of just what we saw so we did see them get into industrial control system environments through that last phase of that Trojan eyes ICS software installer and having folks run that capability for them in these environments the impact was espionage there was a massive amount of sensitive data data stolen off these environments and effectively mapped out the environments if I mapped out your IT enterprise environment that map might not be useful in six months a year definitely not five years in an industrial control system environment we don't change a lot of stuff too often it's unique and weird but once you map it out that map might be good for 5 10 15 years so these espionage efforts have a lot of value in the long term for these operational groups so when we look at it comparable met ICS kill chain we saw the you know we assume that those with reconnaissance done at some point they were going after the right vendor websites right folks there was this weaponization that took place both the spear phishing email the trojan eyes files the waterhole the delivery based off of either the emails the ICS file to the webpages the exploitation was those metasploit modules when they took place sometimes it was something as simple as opening up the files natively the installation right this have X malware itself which then connected up to command and control servers in the action in this case was stealing off data at some point it was just sending out random request to industrial control system ports to see if there was information that could be obtained in other cases it was using a protocol called OPC this open platform communications protocol that was being used to pull off data relevant to the industrial controls some components themselves so from a red team perspective this is probably for most of you very similar to what you do in an IT network the delivery of the exploitation the installation the sea to all of that should sound pretty normal the interesting piece from the ICS focus was number one knowing where you're going to land up with the different methodologies if the defenders go to the web page you're going to land up in the business network of the DMZ if you get the Trojan eyes files they might be more susceptible to getting their supervisory network or control elements compromised testing out to that can help reveal those weaknesses that may be the defenders have the most important piece though was what they did at the action and this to me from have X and the reason I chose it is where you get testable qualities for an industrial control system red team if you look at the pre have X Network ought to be left it's nothing more than wireshark capture of a little Network I stood up and pre infection it was using for 502 and then high level ports on dot 200 doc to hundred was the human machine interface HMI the 30 not 31 32 and 33 were modbus tcp based controllers so there are physical controllers that you could use to do sensing an actuation in an environment 50 tubing the port from modbus tcp if you infect the network with one of the modules of have X specifically the ICS scanner they was module 44 I when you looked at the same environment very quickly you can identify for for a 18 pops up 12 40 one pops up 11 2 3 4 port 102 what was interesting about this is these are all ICS related ports for for a 118 ethernet IP is an example 102 in iso t Sabri step 7 protocol or for the icc p protocol so we start seeing this ICS flavor so the question from red team could easily be if i do nothing more than send send packets to these ICS related ports on the network whether they're open or not could defenders to tech that because in an ICS we don't have a whole lot of logging and host base agents and there's definitely not you know antivirus and a bunch of other stuff running around one of the things that we test and train our active defenders on is if they can use network security monitoring tactics can they monitor the environment to know that dot 30 never uses any other port than 50 to now the question then becomes can my red team understand what the ICS ports are to send out Taylor crafted send packets and an environment that you've already tested and understand that it can support it to these ICS ports to determine if the defenders can see this and more importantly in my opinion if the defenders do see this do they realize there's an ICS tailored component to what you're doing so if a piece of malware gun an environment and sort of scanning around for port 80 I have a different level of response if I think that it might be IT malware but if I'm a defender and I start seeing ICS specific responses I like ICS specific ports even if i'm not using those ones i'm now very concerned you'll also note off to the right that in our requests the malware before it scanned anything just art maps the network and it did it by sending out who has five dot 55 got 105 dot 155 about 205 okay who has dot six top 36 that 106 dot 136 206 it was a slower stealthier way of mapping out the network with the something as simple as mark skin you're not going to drop and you still want to test it right what was going to test what you're doing in lab environments for you anything and ICS you're not going to drop an ICS controller simply by lightly orb scanning the network generally can you do that and can your defenders to detect you since you're not going to be running up is against as many products in the ICS environment not as many solutions and employee agents and things like that the question is can my defenders detect it that active defense component and how long can they detect it and what do they do once they detected so these are all great testable qualities from have X as an example now if we move to black energy to of blackened rd3 this was the third piece of ICS malware that we found in the commune d in 2014 ice I ended up on covering it was targeting not only energy sites but NATO site telecom sites in the other places around the world they named it as the sandworm campaign and it was the same kind of thing right leveraging spear phishing emails to gain access this time there were macro enabled word documents and powerpoints and themes of the powerpoints for energy like oil fracking and power problems there was a zero-day this time especially for the enterprise IT side of house with windows systems but in black energy to there were also ICS exploits specifically for those human machine interfaces connected up to the Internet those internet-connected devices or is going to get pokes in trouble I generally will take showed in and then plug the show to end data into a tool like melty go and try to reveal what those internet-connected devices are for any given facility and show them what their information attacks based looks like as far as their internet connected control systems go the impact of black energy to and black energy three though both were just about remote access Prudential theft in general espionage however we're going to talking about a case coming up with Ukraine where black energy 3 was used as a starting point that then pivoted into an actual attack not espionage but if we look at black energy to and black energy 3 from the industrial control system component it was a pretty targeted campaign now what was useful is and after they landed with emails in the business network the actors would harvest off credentials and then start pivoting through the environment so they were able to make it down not only to the Supervisory control elements but also into the control elements themselves so not only the state F server but down into interfacing with control systems like program logic controllers remote terminal units themselves so there was a greater access once that human component of the campaign was involved to move the actors into the environment this is was usually done through trusted network channels so folks in these ICS environments have to move data from the control elements from the supervisory and sort of skata networks up into the business networks if a power company wants to build customers they have to know how much power is generated and consumed so being able to do that means there's trusted access paths in addition if vendors want to be able to do remote maintenance of this equipment they also have to be able to come in usually through VPNs so for renting perspective in that stage one component being able to not only test out your normal sort of can you detect phishing emails and things like that but can you detect the abuse of trusted pathways those unauthenticated VPN or maybe single access instead of to form authentication VPNs the data historian that's moving information from the Supervisory and fortunately at work into the DMZ can the connections of those systems be accused and can the defenders notice it well the spoiler alert answer is yes all of those connections can be abused and will be but most importantly what architecture passive defenses and active defenses are put in place and test them correctly from that testable qualities so there's a great value right in terms of those spear phishing emails and and customer awareness and education for your users that's always nice we ought to test that stuff out what I want to note here though from the network perspective and I try to focus something differently than FX this time is on the network perspective when plaque energy two and three were environments they would connect out to various servers and do things like an MTP check like network time protocol check but all these server like a 64 server that's just windows like right it's different microsoft update servers which seems kind of ridiculous why am I focusing on this well in industrial control system that works we shouldn't see something as simple as bing com google com Windows Update calm or whatever it is right we shouldn't see generally these network connections in it or derision ating from the industrial control says environments and try to get out so something is simple in a red team environment of sitting on the SCADA Network and doing almost legitimate connections out of the environment two different internet locations to try to test to see if you have connectivity obviously the malware did this to connect to the internet to make sure that it had internet connectivity before it ended up going to its command control servers so without even standing up specific command control servers can you go to legitimate websites in an environment that normally shouldn't be going to these and can the defenders detective it's almost trivial at this point but it's a good check to be able to test your ICS defenders on alright so Ukraine power grid this is a book portion of the presentation this one was a great case because so far we've only talked about the stage one stuff the Ukraine power grid scenario transition in that stage two so what I need you to understand before we talk about it is really just the background on a power grid a power grid at a high level generally has at least three components generation transmission and distribution but it generate through wind thermal you know hydro facilities coal-fired facilities etc nuclear energy then generate power I'm going to transmit this bulk amount of power across large regional locations and then I'm going to step it down and distribute it through towns and commercial cities and places like that you'll also generally have control centers at different levels maybe for transmission sites as well as distribution sites that keep everything flowing and that the control center is also where you'll half of that skata server right the SCADA environment that's watching and monitoring and making sure everything's working correctly it's like the boss in the office right so in Ukraine what happened does on December 23rd 2015 I attackers broke into these facilities they'd already been there for six months actually but it broke in six months before and on December 20 30 2015 they started taking down and disconnecting substations at that distribution level so we're disconnecting substations one by one from these little distribution control centers so they hit the first control center disconnected the substations if the second control center disconnect the substations and this and then hit third one and disconnected those substations there's about 30 minutes apart each and ultimately 225,000 customers were left without power for around six hours until the network engineers and system engineers and ICS engineers and operators could get the power back up but what was important is sort of how they walk through and how they did that so we look at the icsi tracheal chain I'm not going to go over everything the full reports up there in the corner for you and sort of the official report in the community was written by myself Mike asante and Tim Conway released out through sands as well as the DI sac and it details everything with mitigations let's walk through what's most relevant from the red team perspective when the adversaries sort of went after this environment that stage one occurred six months before the actual attack they sense fear phishing emails it was microsoft office documents like powerpoints and word documents that have macros enabled when the users clicked open on the documents it said hey do you want more features you would you like some extra features and all the users like totally want more features clicked enable on macros which of course drop black energy 32 the system and so deliver exploit the exploitation there was the social engineering the installation with black energy 3 the command control servers that had connected up to race who connected up to various command control servers and the action in this was allowing remote access and credential theft so the attackers moved in pivot around the environment and looked for credentials that can be leveraged against the ICS networks most importantly they looked from VPNs from the business networks into the industrial control system environments and into the control environments through these control centers so that allow them to pivot to that stage to what they did in as they were wrapping up the stage one over the course of six months is they made sort of their inventory of the environment they figured out what the environment was using it figured out the different control environments the different systems on the network they realized that there was three different distribution management systems so three different environments that were spurred out across these three different facilities to go to control the substations can they develop knowledge about an interface with each one of these so that they could use the system against itself to remotely disconnect the substations when they developed that knowledge they would have had to test they were doing at some point but we know a little bit more about what they tested because they also found what's called a serial to ethernet device basically they found equipment that acted as a bridge between the control center and the remote substations out across different parts portions of Ukraine a serial to ethernet devices take the control center IP protocols and data and translate it into serial protocols for the control elements at the substations so they found what those were and they develop specific malicious firmware against all of these different little around 57 of these little serial to ethernet devices so that when they attacked Amy that attainment came to nine to do the attack when they ultimately took the substations offline and disconnected them they also pricked the serial to ethernet devices so there's no way to remotely bring them up but they also delivered some other good and stuff for the actor of the defenders not only could you say that deliver themselves they add the remote desktop access to the human machine interface it's a remote desktop assistant was enabled but they also delivered a piece of malware called kill disk across the windows environment of the SCADA system around nine hundred different systems in fact where they deployed this kill disk malware which upon reboot would delete the Master Boot Record or logs off of the environment they also modified the network ups and when the power went out across Ukraine made sure that power went out across the control centers as well after rebooting the systems to trigger kill desk and they draw the network interface card on the UPS so that if you did get up some systems you wouldn't be able to remotely go back in and turn on UPS or turn on any of the systems or unschedule what they had scheduled so the execution of the ICS attack itself was executing the firmware attack executing the killa disk attack but also actually disconnecting a substation is through learning how to use the ICS now this is an important concept here it wasn't that the attackers used malware now we're alone would have done much these environments we took out the skate environment the power grid would still work it's not ideal you lose automation but the power and lights don't just go out because the skate environment without what was important here is the attackers learn how to use the industrial control system environment against itself black energy 3 enabled the access kill disk amplified the effect but it was really the knowledge of the attackers in the industrial control system environments that allow them to disconnect the substations in the first place so from a red team perspective things that I think is very important to take away is getting intimately familiar with that ICS and thinking about how it would be turned on itself how would you sit down and use the industrial control system biomet against itself when you're sending legitimate commands there's no a bee and point age and whatever else is going to stop those legitimate max because they're buying nature legitimate so an attacker issues a legitimate command about disconnecting a substation which an operator might need to do at some point there's no product that then stops that so can you do that what would it look like and how would defenders respond is there a sort of cyber readiness level of the ICS a good red team take away in my opinion here and a good thing to teach the defenders would be hey look you know the first control center and your network gets hit the second control center across different region gets hit well before the third one is delivered do you really need internet access right now do you really need a remote desktop assistant enabled can we have a temporary hardening profile for certain features that we turn off that facilitate better defense and a better response sort of posturing before the attack and to be able to test that as a red team again very very important so what are some of those detestable qualities new crane well we saw that remote VPN used to be taken advantage of in an ICS environment VPNs are common however it's not common for it to be high spikes and usage that's also not common to be leveraged for a couple minutes at a time if someone's coming in to do remote maintenance you might expect them to be there for two three four hours at a time not ten minutes on the network then jump off so from a defender perspective they'd be needing to monitor the VPN so from any red team perspective can you do something that won't affect the ICS at all you just start logging into BP ends you just do it more comment more often can you do it random session links can you do this and have them notice you is that something that they would notice at all the firmware updates there are no reasons out there that firmware update should be taken place during operations in other words the power grid operators if the engineers need to do a firmware update it's going to be in a maintenance period where the station is down right it's only done while power is being transmitted and distributed so could you see this on the network could you stop it could you ensure that only specific systems are allowed to do for more updates and only certain times of years when their maintenance periods remote desktop because you identify that the remote desktop assistant is not needed at all all the time and that it definitely should be used at every single system across three control center simultaneously the harder one is legitimate commands and this to me is really an aspect of traceability even if the legitimate commands can get issued at that end of the ICS cyber kill chain can the operators go back after the fact to log as information to know exactly what commands ratio you didn't win so from a red team perspective it's something useful to be able to do now the testing of that gets very difficult and this is going to get where we'd talk about recommendations so let me get to the are sort of last slide here these recommendations for folks I'm sorry sorry I jumped ahead have one one more slide for the last one I want to note that most of the stage one stuff is traditionally your Windows environment enterprise network may be getting into the DMZ the Supervisory network we talked about stage to attack it's really that level 21 and 0 that control elements the physical equipment so a lot of your red team skills translate very well into stage one the focus though is on what can you learn about the ICS including pathways and access points to get into from there the attack the ability to cause an impact is really engineering it sort of ICS knowledge so my recommendation as we end this out and then open up to questions number one safety and reliability are paramount but that requires coordination the people that will authorize you to take actions may not actually have coordinated they should hopefully but may not have actually coordinated correctly with everyone on site I really don't like the idea of a blind red team assessment of an ICS environment there's too many things that can go wrong with physical repercussions or Consequences so I look for coordination and I look to be able to test those defenders since I'm not too concerned about the products on the network testing those I might be for gaining access to the environment to VPN the tunneling but there's not so many products in the ICS itself what I really care about on the red team assessment that counterintelligence focus is on my active defenders can they see it will they detect it how long do they detect it and what do they do when they do so it's okay to have coordinated and just see if they can actually run through and see this I usually ask for my red team's to start off of the tabletop exercise it's very useful to sit down and ask the questions first it not only allows you to understand where the weaknesses might be and what things need to focus on but also do a challenge different notions your ICS asset owner might say well we're not connected to the internet or we don't have remote VPNs and so you couldn't get access anyways and then to be able to show that you can is a very important takeaway for them to note them they don't have the visibility across the ICS they thought they did you know a good question might be could you see this if it occurred to you I've had facilities already and we've started seeing red team's against ICS environments mimicking the Ukraine attack and say if the Ukraine attack happened against you same style the methodologies the tactics the techniques the procedure is being the same what impact could achieve could it happen to you and what would you do how would you respond being able to roll down that almost in sort of a check listed out function is a really valuable takeaway for a red team for an ICS environment but it's not about throwing metasploit modules it's not about seeing if you can get a shell on the PLC and it's not about seeing if you can you know do DNS exfil off of a control system because those aren't d tactics that we're seeing from Red Team perspective those are things that might be forward leaning we want to get there we want to see cool things and then help out but being informed by what's actually happening in the ICS environment what we generally see is a lot of stage one activity to learn the ICS and then using the ICS against itself learning the ICS environment so well and persisting for so long that you can build up the knowledge to bring it down that you know the physical safety systems that are in the way from the design engineering documents that would stop over pressurization of a certain pipeline you can answer these type of questions from learning the environment now the other thing that I'll sort of leave you with this too is it's obviously very very critical to test these things out finding a lab network if possible a maintenance period some an environment where you can go and do all of this when it's not active operations is going to be extremely important because you go definitely don't want to be the guy or gal that takes down portion of the power grid testing around on systems that were there for sort of purpose of defending and training so i'll give you one story and then we'll open up the questions of sort of how this can go how important it is to learn the ICS so i got called in one time specifically to more or less babysit event duster and I was like what's in there like we need you the facility said we need you to come in and just walk around make sure this guy doesn't destroy anything so dope okay I'm just probably qualified and you have to call him in and they're like nah we had to have an annual pen test is gonna be valuable we'll take it seriously but I just need make sure he doesn't destroy getting make sure whatever that's fine I had nothing to do and it was local to my town and I kind of like infrastructure water power you know oil etc so if all this guy round he's doing everything okay this very IT centric wasn't doing a lot of Kauai CS stuff wasn't really testing out the adversary capabilities it was it was a pen test not red team but he was good so he gets to the sort of meeting and you know being the exercise into is like two weeks there he goes and presents and he's got everybody in the boardroom right it's it's the CSO it's the chief engineering as a lot of folks that are excited about seeing what happened and this guy is eating it up I'm he is arrogant and I know none of you have ever met an arrogant pen tester or defender before nobody arrogant possibly in any any role of security this dude was arrogant and he's like oh yeah I mean so I could have totally taken down this whole operation I mean like slips around his laptop and he has a two machine interface top that he actually had remote access to like I push this button right here boom there must just start to look around like is this guy serious an elite engineer without missing a beat says push it and I'm like low hold on cuz like hey you know I don't really believe this guy that is going to blow up something from pushing the button but hey we shouldn't test it on live equipment in the engineers like now budget and chief information security officer does not hold the power in that situation right chief of engineering that lead engineer is is the boss in that situation and the pin dusters like you know start to protest a little bit and the engineer in Axum on finally has a screw it and pushes it nothing happens without missing a beat the engineer doesn't boom it goes well Sonny and this was as degrading as you could possibly imagine with the engineer applied well Sonny what you're in map scans didn't reveal is we've got these vowels on the pipeline that if you try to over pressurize it it just opens up the physical valve and we just let off all that hot air that's a cool analyst news pretty good burn to the meeting but the point in saying this the reason I bring it up is those physical drawings that understanding of the physical safety systems the over pressurization valves things like that aren't going to be revealed on that I piece can and sometimes even in mapping and ICS network can go Fort Lee wrong and take down sets of equipment anyway the point is staying a stealthy manner learning from the adversary tactics that we've seen the community and understanding the ICS as well as its physical components and what can go wrong and can be tested out safely to show that maybe he operators engineers or defenders don't fully understand their environment that is the value in a good red team that is the value in testing out those defenses and their best defenses aren't the firewalls their best offenses are other people and showing that opportunity for education with their people so with that I'm going to turn over we've got just a couple minutes for questions I we got to get out a little bit early for the webcast day I know it's that Dean of the day for most of you so I appreciate that sticking around but feel free to start asking questions now I've got a couple so I'll start the first one the first thing is what's the worst thing I've ever seen happen with a red team assessment and nothing in terms of loss of life or limb it's not a scary right they design these systems to be safe from a safety perspective even if they don't design them to be secure and we definitely got do better in that area but I have seen you know pen testers red team members come in and take down systems right to do an in maps can and didn't understand that sending random packets and scans across the network could stop sensitive controllers and so I've seen million-dollar impacts off of stupid scans and environments and and it costs the facility greatly so insurance is a really big thing for red team's of course but also getting good authorization on what you can and can't do in those environments and I got a question here did they catch the Ukraine guys yeah so they didn't right now the understanding in Ukraine sort of what's been on tiffa gated by the community and who the sandworm team is is that it was the the Russian government I actually don't agree I don't think we have enough evidence to say that however the attribution sort of motivation and what was achieved and timing and location and things like that that my lines so I think that the Russian government at some level may known about it may be funded it but I don't know that we can just say government most people think of funded team and they think governments but we're seeing more and more companies do these things as well you know I t hacking team gamma group different companies around the world that are selling you know selling these other capabilities selling operations we've seen them sell things that governments before I I think there is a a high chance and again we don't have a whole lot evidence around this I think there's a high chance that maybe a funded team was carrying out operations at the request of a government potentially russia but i didn't i didn't find enough in terms of what we saw from the evidence to defend elysees it had to have been the Russian government I think it occurred out of Russia though and I cannot imagine a scenario where operations were being conducted against civilian infrastructure out of the Russian border and the government didn't know about it I just I it it would be unfathomable to me that that could occur without the Kremlin's knowledge so I don't I don't think we let them off the hook in terms of the international community I think that there's geopolitical sort of pressures and responses and things that we have to do things that were definitely missed opportunities in my opinion on on coming out and making a big stand about how attacking to the infrastructure is not okay but there was not enough proof in my mind to actually say definitively this is any given government but again I think that's part of the problem so we got a response from Steven you seem to disagree with me feel free to follow up on why you disagree but I worked the case there wasn't enough to save as a Russian government so see we got another question here how can we set up a basic lab or ICS cater to start a good one rubit one what I like to see is not a huge investment in tens of thousand dollars for the control systems that's usually where a lot of people start I mean let me save a bunch of money and try to figure out ways to buy a bunch of physical control systems it's useful what I like to see instead is start small i would say grab some you know raspberry PI's and set them up to run modbus tcp clients and see how you can air base with it what you can learn then move up and grab some equipment may be off ebay what most folks will tell you is it's not just about the control system that are physical it's about the applications that run them and that can be the most expensive part sometimes so when you're looking for good rugged equipment that has software that i would consider obtainable and not in the tens of thousands of dollars you know rslogix phoenix contact and these piece of equipment you know phoenix Siemens and and rslogix all have a couple hundred dollar versions of control systems that you could interface with learn the logic on CL ladder logic works and start interfacing with them on the education piece obviously we teach that out in the ICS curriculum side about how to build and and security systems and defend them but on getting started the basic lab I'd start small raspberry PI's move up to ebay dive equipment move up to control systems that you buy from the vendor and just get them networked together build out a small lab based off that Purdue structure that I showed you so Google the Purdue model can build out like a domain controller for the business network with some windows BMS build out a DMZ into a plant environment where you've got some software that's interfacing with the control system and start seeing how you can end test and move around and get to working with those all right that's actually all the questions that came through which is ending us about covenants early which is useful I know most of you were hanging at the end of a long day so I appreciate you hanging in there and thank you so much for attending I would leave you with a little body comics these are comics that always right to keep my sanity and information securities i'll leave you with one of them put my twitter handle down there as well as the comic i would also encourage you to follow the the sands pen test group especially for hackfest is going to be a whole heck of a lot of fun i am extremely jealous that i can't make it this year we have one last question we have a couple of questions coming in the very end what if you encounter an air gap there are some facilities that actually do have an air gap I know it's mostly a myth in the community people think they have an air gap they don't at best the absolute of a high latency network but generally there's different ways to move through it you SBS but also there's a lot of times I think they have an air gap they just don't but if you actually do find an environment where they have it look to see where you have data moving in and out of the environment they have to at some point it might be USB is it might be data diodes but there's data flowing in and out of the environment and also look for we're sensitive ICS data is stored outside of the xes you might have system engineering drawings and documents and sensitive ICS related data in the dmz that's not part of the air gap and but generally speaking the air gap feels pretty commonly in the community because data access is needed in the boardroom this is you know pushing that into the audit you know into the community to be able to have more and more visibility to access and then the last one to see and yet so replay of this is available this and the slides will be available afterwards Carol's going to take it from here so care if you wouldn't mind go ahead and closes on out all right thank you so much Robert for your great presentation which helps bring this content to the sands community to our audience we greatly appreciate you listening in for a schedule of all upcoming and archived sans webcasts including this one you can visit sand org forward slash webcasts until next time take care and we hope to have you back again for the next sands webcast

Channel: SANS Offensive Operations

Views: 1,356

Rating: 5 out of 5

Keywords: ICS, SCADA, Red Team, Penetration Testing, Pen Testing, Penetration Tester, Hacking, Hacker, SANS Institute, Ed Skoudis, John Strand, Robert M. Lee, Metasploit, Scapy, Nmap

Id: ERnPuGvH_O0

Length: 52min 56sec (3176 seconds)

Published: Fri Aug 19 2016