Why Most Cyber Security Training Fails and What We Can Do About it

thank you everybody for attending this talk it sort of quickly introduce myself I'm a ruin vishwanadh I'm a faculty at the University at Buffalo and most recently I'm also faculty associate at Harvard's berkland client where Bruce Reiner is so looking forward to working with him this year I want to give you a little bit of a background of what I do what I've been doing for the last decade and a half of my life for for a while now as an academic researcher my focus was on how hackers terrorists cybertor is called them what what you may affect breaches and how they do this by compromising people alright so and I've been working in this area of technology use technology adoption for close to more than a decade put out a lot of publications I don't want to bore you with the list but there's a long list which basically only the academic community cared about so the longest time my paradigm of working was to actually create attacks and to study why people fell for different kinds of attacks be they social engineering attacks be the different kinds of you know spoofs and breaches and so on and so forth my focus was on undoing these attacks and studying why answering that why rather than just showing that it can be done like I said I spent a lot of my time doing that and most of those work was for the academic community and no one really cared much about it outside of that community then I like to say came King John and what Kim did thanks to Seth Warren's movie about him was you know he decided to breach Sony Pictures and suddenly thanks to that bridge there was now a real visible outcome that people started seeing there were jobs people were losing their job millions of dollars were being spent in trying to you know remediate these breaches and suddenly I was all over the place started writing on CNN quite a bit and at every point along this way I started trying to tell people to stop focusing on the breach and the salacious gossip that was coming out and start looking at what this really meant and each breach starting with Sony Pictures followed by Ashley Madison followed by Target followed by OPM followed by infrastructure attacks they kept getting bigger and bigger and bigger and I kept writing about this and for a long time I was the only one trying to build attention to how these breaches were happening let alone saying hey you know we got a problem here and the problem is we gotta start studying these users and understand how these guys are getting into these networks let's not look at the outcome alone let's look at the how right and we had the DNC hack we have and this seems to be an ongoing process that is never ending with all of these one of the common threads that was weaving through this was spear fishing and spear fishing is something I'm going to talk about today although my research looks at other types of breaches as well but spear fishing was starting to get everywhere it was kind of so common that almost three-quarters of all breaches were using this particular vector so this became the common the most favorite vector out there when it came to affecting breaches when it came to getting a foothold into a network and these guys were hitting every industry you know thanks to thanks to people like King John Putin you know every hacker out there every hacktivist out there breaches were everywhere in a we thought education was not getting breached by 2016-2017 education was getting hit every industry was starting to get hit with spear phishing attacks and what you constantly saw was that the attacks were getting more and more consequential so in Sony Pictures you had the leak of emails in target that was leaks of passwords and it kept getting bigger and bigger and bigger and every time when you looked at the cost of remediation it was keeping on going up just to give you an example last month in Buffalo New York where I am we had a hospital that was hit by a ransomware attack they refused to pay the ransom they spent fifteen million dollars in remediating that attack which lasted for all of like a month so the hospital networks were all done for a month people are losing their jobs things are getting more consequential and arguably you know even the ransomware attacks now we still don't know exactly how these came in but spear-phishing had a hand or a bar or a something to do with the passage of these attacks and if we just track the scope of these attacks the speed with which they've been happening we had 300,000 infections in 150 countries in less than 48 hours and I wrote a piece about this on CNN saying this is the first global technological pandemic that we had and this was a huge deal because now the paradigm has changed its global and if this were a disease every one of us would be really really more concerned about this than we are and so everybody is on this though right everybody's perpetrating this you have activist groups out there you have the PLA working there you have the Russians doing it you have local guys doing it and all along the target is the user right the employee in the organization the end user out there they have become the weakest links and we like to say this all the time you're the weakest link you're the weakest link so what do we do about these weakest links I mean that's what I'm going to talk about today so in general there are essentially three approaches that almost every organization out there or everybody out there who cares about this all over the world and I I've worked with people in the EU I've worked with people in Asia in Singapore and there's basically three approaches that it comes down to and I call it the three C's the first C is that is a technical approach right containment so we've tried to contain people within their networks this is how we deal with users the second thing is where we try to constrain them right this is also a technical approach right we air-gap and and all of these are relatively successful there's not one answer here although everybody is looking for that one answer these have all been relatively successful the third approach the one I'm going to talk about and I just want to talk a little quick about how successful these approaches are people find workarounds okay this is the constant problem with with with users today everybody is constantly trying to be one step above security folks and when I talk to the security folks they're constantly lamenting how users are not paying attention not listening trying to find a workaround and I know of people who work in government in insecurity who don't do patch upgrades or don't upgrade their OS on their mobile devices because their children are watching Netflix on it and they don't want to do an always upgrade because it's gonna stop their ability to watch so you know I collect these stories and I talk about what challenges out there when it comes to dealing with users so the third approach which is what I'm going to talk about today is a very much training all right and there is a an industry out there now work close to a billion dollars of various awareness education training programs that they're constantly hitting people with okay this is right now one of the most popular perhaps the most proactive approach out there when it comes to dealing with the user the weakest link in our networks and this is called the convincing approach right we try to convince users and broadly this approach takes one of two forms the first form is what is what I like to call as didactic training that active training is where you get the user in you focus your energies on teaching them how to do things better how to be safe how to use you know email see if there are programs out there companies out there that do this this is a focused approach you spend a day maybe a couple of days trying to train them to get smart to be like us to be you know cyber secure resilient and all that stuff some of this you know he's done on an onboarding basis but this is an approach that's out there quite a bit the second approach is what is the embedded training approach and what is this this is where you know you're you're teaching people by making them make mistakes and I'm going to talk about both of these approaches and I want to talk about what we have seen when it came to the success of these approaches because there really isn't much data out there about how successful each of these approaches are other than the data coming from the software the companies the training companies that are putting them out so you know I am very skeptical about that data sometimes so the first one that active training this is this is that focused training that we spend a lot of money doing when you look at the the overall success some of the academic research on this comes from we are on 2004 2005 so it's a little dated and one of the first studies this is a very highly you know commonly cited study was done at the US Army Institute at West Point where basically what they did is they took about 500 you is similar to Cadets they trained them for about a day and then they fished them a week later and what they found is by the end of 24 hours they'd forgotten the training they all fell for it all right 80 percent of them fell for it very high numbers very problematic now again this is 2004 2005 so my research team and I we decided to replicate this we said let's do this but let's do this right let's do this with you know a different control group and so we did this study we took a large financial company in the United States it's it's a pretty large company we took about four hundred of the Templars who are all from the same who are all playing the same role in the organization they're distributed across the United States we took half of them and we trained them using off-the-shelf software that I will not name very highly rated and then we fish them 4 weeks later and one of the things that we did is we check to see that we didn't fish them every day we fish them on three non-consecutive days and there's a lot of reasons for it some of the literature some of the the data that we see is it on which day that attack comes in like some for some reason people are just more likely to click on things on a Friday as against on a Monday because it's the first day of the week and so on and so forth so we wanted to get that data right and we fished them on three non-consecutive days and what you quickly start seeing is when you look at day one and I'm just going to show you the day one data by the way this is the attack we used this is a very simple attack we just copied a Google Drive page and just we sent that attack out okay when we sent this attack out we were looking at some of the data that's out there Verizon's data breach investigative reports they talked about 50% of all clicks happening within the first 24 hours or within the first hour actually and when we looked at our data it matched that so within the first 24 hours the vast majority of the clicks were happening very very quickly people would just it didn't matter it was a Monday and they were just hitting it this was very interesting because we had two groups in this we had a group that was highly trained and a group that was not trained and so when we looked at the people who were highly trained it was absolutely identical with a little bit of a difference it was statistically the same in other words the training was not really having the effect that people think that it is right so we were saying wait a minute didactic training doesn't seem to really do it although when we looked at the final data you know about 32% of the people who were trained clicked 35% of the click people who were not trained clicked okay so there was a big 70% that didn't do anything that's you know good news but 35% is a lot ok 35% is real bad news if you look at it that way but the training effect was marginal actually it was non significant it didn't matter so in other words you know do something train don't train you're gonna get the same results at least when it comes to didactic training from what we looked at okay so we said okay let's look at and better training so what is it better training this is the more popular training this is what's out there today many companies many of you probably work in companies that actually do this right this is the kind of training where you test people right and you constantly test them and then every time so basically you know you have a person in the company who does a phishing test right some of you sitting here do that you've faced this test you get this test you repeat this test often enough at varying intervals and then every time you click on a link something pops up and says hey wait a minute you just clicked on a link you got finished right you're the weakest like and here's why and we train you and this is a this is an interesting approach and and if you fall for the fish once they train you in some companies you fall repeatedly they've warned you in some companies they embarrass the hell out of you there's a lot of different approaches to this that's you know it's open territory what you want to do to make this stick but this is the approach that's more common out there this is more popular most companies buy into this I call it and are trying to learn to drive and by making mistakes so you know it'll try to learn to drive throw a stop sign you then show different signs this is like trying to learn driving and you keep on doing this till the person becomes an expert driver right hopefully at some point these persons are gonna have accidents anymore that's the approach it's a great approach I think it works to some extent so how effective is this mirror did a study this is from 2015 this is a commonly cited study and we wanted to replicate it so in the mirror study they did three trials so they took three different attacks on about 1,500 people they found that 35 to 60 percent of the people fell for the attack regardless of the training so once again the training did not have an effect what was interesting about their results as they found three groups one group all clickers they click on anything that's out there all right 11 percent that's a big 11 percent it's really problematic if you're sitting if you're an info circular like Jesus Christ that's a lot you have the 22% non clickers who don't click on anything now if you're a security guy you're like wow that's good but if you're in a company that's looking at productivity what are these guys doing that's probably not an opening email I don't know what's going on there my problem as an academic how look at that random 67 percent and I say we're screwed we have no explanation for that we don't know what these guys are doing they're just randomly clicking on things if I know you click on everything I can take you aside if I know you don't click on anything I can say all right this person's not a security risk but if you don't click on anything and you're just randomly doing stuff I gotta figure out what's going on so we looked at some of this data ourselves we looked at a company that provided us some of this data their data so this was a company that provided data of 20 spear phishing attacks over a two year period so we up the game quite a bit one from three we went to 20 and this is a popular spearfishing program out there right anti spearfishing program each of them were followed by embedded training popular product which you've probably heard of and these were the results we got and here the results are a little bit better but they're still very problematic if you look at that second column there 24% of the people were clicking on at least one attack and this was over a two year period of 20 trials so if you do the math they're around 30% of the people in this company after two years of being hit repeatedly we're still clicking on links in other words still falling for a spear fish after spending all that money all the time all that energy we're still not getting where we want to be getting and when you look talk to sis ozon and then 4 sec people who are running this show they're always perplexed they're annoyed by this you know they'll say things like and I can't believe these guys I don't we train the hell out of them they don't get it they'll say things like you know we should fire these guys Admiral Rogers at the NSA said we should court-martial people who fall spear-phishing test it's it's we should embarrass them some people shame them out if you're a little bit of a pessimist you start saying things like you know we need to learn to accept that people always gonna be a problem if you're optimist you start saying things like I think we can just keep doing this and it'll get better right we've gotten to 65% maybe we just keep on doing it keep on doing it and then they get better over time and get better over time and better unfortunately if we really want to solve the problem we gotta stop just saying people fall for an attack and that over time it's gonna get better we got to figure out why I mean why is this happening right as Alex Tamils yesterday said you know we're very good at showing that there is a problem but we're not very good at explaining or correcting it and to correct it we gotta get to the how and the why why are people falling for it and the answers are important because right now the cost of training is expected to increase the most it's exceeding any other form of technical solutions if you looked at that three seas containment and constraint is actually far more inexpensive compared to training training is gonna exceed all of these costs and we don't even factor in the time lost in training the ineffectiveness of some of these approaches and so when you look at the answers though the answers are right in front of us so one of the things we did is we did our studies where we fished people repeatedly and unlike other studies we went to the bottom of why and when we ask the questions the answers were right there people would say things like I clicked on it inadvertently without thinking or they would say things like you know I clicked because it was the week before Halloween so I got a Halloween fish I clicked because it seemed very suspicious so I emailed the person back very rational answers actually if you really think about it or they would say something like I didn't think it was a big deal I'm just clicking on I got damn bling do it all the time um I was careful to open it on my phone these answers actually tell us a story and the story it tells us that what we think of as a people problem it's not really a people problem it's an understanding of people problem we don't get it we haven't spent time answering the why and so one of the things we did and when I say we my research deivanai is over a Peter of the last you know how decade we've been constantly doing these tests but we've been trying to do come to the answer of why so that we can build an understanding of why these things happen and how do we make things better and that's what I'm gonna talk about right now so why do people fall for spearfishing what is our research tell us the first thing that we are understand is all the people out there in companies today who are constantly phishing people using these phishing software to try to make people more resilient you kind of have a mechanism to understand what's a good attack because you find keep fishing you every day or every month twenty times over a period of two years and I keep varying the attack and make attacks looks very complex or very easy make it so easy for you that you fall for it make it so complex that you never fall for it I need to be able to tell you what's a good attack what's the standard of a decent attack and for that we realized that there were only basically three things that we needed to understand and what we call that is is there's a model that we developed called a suspicion cognition automaticity model called scam it's it's a published peer review piece and we started by looking at what's a good attack and we came up with the triad and the triad we use the word called vishwas which is a Sanskrit word for trust it's a trust fired the trust crier the vishwas tried and it consists of three parts right what you need is you need a trusted source so a good phishing attack has a trusted source and there are tons of trust sources out there right Google is a great trusted source Amazon's a trusted source PayPal's a trusted source you need a modifiable field you have to have a field in that attack that is so easily modified and it's constantly something that gets modified to the end-user so the end user is not paying that much attention to it and you can see it right here this is a good attack we modified the URL we modified you know the cell the provider name and a good attack as a third part to it and that is some kind of user routines some kind of user expectation if we could get these three things right we can match the user to the attack I can get into any network and and when we have done this we can get a 50% success rate if you can get a good enough trustable provider and do this so here's an example of gizmo DOS test if you guys have been watching reading the stories on this gizmo did a non scientific test well basically they sent a phishing email to 15 people in the Trump administration and what they found is roughly half of them clicked on the link in fact two people responded to the fish okay now was it a good attack yes look at what they did they took an attack from a very credible source a trustworthy source a modifiable field and what do people do when they get a Google documents link they click on it user routines all these people expected it so the 50% success rate on the other hand if you did an attack like this in those 20 attacks that we reviewed from the company that I was talking about one of them was this 0.2 percent success rate of course there's nothing here you gotta be crazy if you click on this right but one person did so but we used this right and the bad guys out there are very smart about it they use it so this was the letter I got because of the anthem attack giving me credit monitoring just to give you an idea this attack happened I think in February I got this letter in November by mail because never being very careful but the next day after the attack I got this and what do you think everybody was doing they were all clicking on that because it was something people expected nobody thought anthem would wait nine months to send you a letter that's a good attack and there are many such attacks that we see and if we can compare the good attacks we can understand what's going on the second part was understanding why people click so we understood that there's something about the attack that we need to match to the user expectation then we were looking to see what is it about people I can we come up with a quantitative model where I can say you know what here are a few things that you need to measure and if we have done this over a period of you know 5 to 6 years doing repeated attacks all over the world you know within companies that have allowed us from her giving us permission to do this within university settings and we realized that there's basically only three things that we needed to measure one was was there something that triggered the end-users suspicion the second was there was something about how that person was thinking and the third was there was something about how that person was doing and if we could capture these three things we can actually predict who's gonna fall for an attack before we send that fair shot so that was the goal of this process that we engaged in and so I'm going to quickly explain to you what each of these things really mean right so first things first I'm going to talk about cognition right what we found is how do users think becomes pretty important and what do users do use there's a cognitive misers what's a cognitive misers they're not spending time thinking they're making the fastest decision with the least amount of information and the least amount of effort all right and so they use cognitive shortcuts there is a huge body of published literature on this and the cognitive shortcuts people use are triggered by something in that email generally a logo or trust indicator something that quickly tells me what is this that I have to do and I'm gonna do it and if it's part of my routine I'm just going to do it here's an example every time I go to Starbucks I get to click on a link the Starbucks logo gives me credibility so we tried this attack and we found it was very successful for some reason people thought Starbucks and Google because they're part of that same Google Fiber thing people gave us their logins and passwords it's very easy to do right because they're using a shortcut and we took advantage of that shortcut and our success rate was around 60% and there are many examples of this here's that here's an example we would be created for different Facebook pages and for those of you out there you can see there's a missing e there's a missing there's a different color there's a missing Yi and a different color and then there's the original login page which one do you think is the most credible here anybody which one do you think you were to open or to put in your login and password anybody this one does everybody think it's this one right this one is the most credible isn't it would you put your login and password in here I wouldn't all of them are spoofs right how difficult is it to do this I just copied and pasted the whole thing right so if you're using a shortcut and you're thinking trying to out think the process i canäôt think you because without a URL you don't know what's going on here right so but what we realized which is pretty interesting which this never mattered neither did this because we did this study in October and a lot of people said oh we thought it was Breast Cancer Awareness Month and they changed their color I don't even have to work hard at this you know it's it's easier to be a bad guy because people just give you their data the other thing we were measuring what do people believe and this is a real interesting data and I can tell you this we've done this we've asked these two questions now in in Asia we've asked it in the United States we've asked it in Europe in different parts of Europe and I can predict the answer the first question we ask is how easy is it for you to spot a spear phishing email and these are employees 600 employees in a company and everybody thinks oh somewhat easy very easy I can do it I'm pretty good at this and then we ask them a follow-up question how easy is it for any other employee in your company to do this I look at this everybody is a ninja in their mind right everybody thinks they're better than everybody else and this data is so predictable we did this exact same thing in the Netherlands same answers how easy is it for you somewhat easy how easy is it for somebody else somewhat difficult and then when you look at the training data and you say how much time did this person really spend looking at your training some of them are spending less than 30 seconds on it because they're geniuses they know this stuff they don't take it seriously that's a big problem right you need to be able to convince this portion to first pay attention to what we're giving them the other thing and this is something a lot of us suffer from is what I call the dunning-kruger effect I don't know if you guys know who Dunning cook of what dunning-kruger effect is but this is a this is a real story this happens in Pittsburgh I know you're from Pittsburgh and there was a guy by the name of McArthur wheeler real name who you know broke into a bank and before he broke into the bank he applied lemon juice on his face because he had read somewhere that lemon juice makes you invisible because it makes ink invisible so he figured the digital cameras he's got to be invisible to it I can't make this stuff up okay that's that's all real the story is so when they caught him he kept saying how the hell did you guys catch us and he said well we were on the camera I said how'd the camera catch me lemon juice on my face so two of these guys Dunning and Kruger wrote a paper about this and they said what makes people think that way right there has to be a hubris about us or a lack of knowledge and we did the same thing with technology which cybersecurity and what we found is people have ideas in their mind all of you do I do you got a snout you know as the end user out there what's the safe word document a PDF document or a Word document invariably 80% of them will pick PDF and ask them why well I can't edit it which means it's safe right we think we know and there are lots of this right when you ask people you know what's up safer email client you know what's a better operating system you know what's a better Wi-Fi network and watch more secure everybody in their mind is carrying these ideas of security most of which are not based on fact and the problem with that is with these risk beliefs people who have low risk beliefs tend to be very confident so you think you know when you really don't know and no one's told you you don't know yet so you go about doing what you're doing and that's how you get the macarthur reader of fact right you wear a helmet and you stand on top of them in a tightrope because you think the helmets gonna protect your head because that's all that matters right your head but the problem is you don't know what's the risk there and so this is a problem right this is something that we've been trying to measure the other is habits automaticity people have habits and this is checking email and you know checking email when you're walking when you're talking and a lot of these things make you click on things when you're not really thinking it to you're doing it really fast and what we have found is it's easier for me to get you to click on a link if you're on a mobile device it's a lot easier for me to do that ok so we put these things together right we put these for ideas together and said let's try to measure this and come up with the best ways of measuring this and we realized all we really needed to measure when it came to this was a few different things and this is an empirical model and it basically consists of five five and six different constructs and we have a measure for it okay and this measure is what we call as a cyber risk index so what we found is if I can ask you questions basically comes down to 40 questions we're able to come up with the individualized score that ranges from 0 to 100 that tells me your risk your likelihood of clicking on one of these spearfishing links okay it's an individualized score so each person gets a score and you can use that score and change it and get it updated over time so it's not a static score and all you need is a person to answer 40 questions in a survey okay and it's very quick to do and you can do this using whatever software you're using so if you're using a package right now to you know an embedded training package you can fish people do the survey and then fish them again and see if the data works out so we've been able to do this to match and to refine our measurement to say you know how good is this and then we've been able to now aggregate it across division so I can take the score just like a credit score and make a risk score for that division for that group for all the employees in that area because right now there is no metric out there right just saying how many people fell for a phishing attack or a simulation doesn't do it we need to be able to say why and we need to be able to say what's the quantitative threshold error and so what we have done is we've then come up with mechanisms of taking that index and the answers that gives us so the index not only gives me a number for each person it tells me why you're gonna fall for that attack so in other words is your risk beliefs problematic the index tells me okay here's where that number is coming from primarily it's coming from their risk index or it's coming from their risk beliefs that way when I'm training you or whatever I'm trying to do that change can be focused on the problem rather than basically just keeping on hitting you with more and more simulated attacks and we have done this where we have developed a flow and we said okay what is the index telling us what is this person suffering from so the analogy I use is if you go to the doctor the first thing the doctor does is he diagnoses you and what are 40 questions help us do is it helps us diagnose what's ailing the patient because right now the approaches you go to the doctor he throws a pill at you you take the pill you come back sick he throws more pills at you and then you keep on doing that and then you keep blaming the patient for not getting better it doesn't work on the other hand we're able to pinpoint why and then say okay here are the answers and for some people they may be bad habits that they have come up with for other people it may be their risk believes for some other people it may be a combination of factors and now I can come up with a solution based on what that person ails from okay and that's what we're using this index what is really interesting is when we did this across companies and you're still collecting data on this we're still trying to refine it and make it better and make it sharper what we found is that there is a tipping point to that 0 to 100 number okay there is a tipping point to it and the tipping point is around 68 to 75 so you have a score from 0 to 100 when you get between 68 and 75 what we find is that people's resilience actually goes up right so we've actually studied this we are actually implemented this and looked at it and what we notice is for people whose scores are about that threshold of 68 to 75 you start fishing them they don't fall for it anymore okay so the job the way we look at it is we want to bring people in companies to get beyond that 68 to 75 threshold does that make sense so that's our index and right now what we're doing is we're trying to figure out ways in which we can refine this even better right we're trying to make it such that it becomes easier to implement it because all it is is 40 questions and it gives you a diagnostic that is you know a agnostic to the device so for instance a lot of training today is with the no use case that is very limited so you're training people to work in a Microsoft environment and on a desktop environment many of these environments are changing and don't even exist anymore many people are bringing devices to work many people are in different environments and whatever training we are doing is so siloed so limited and yet so ineffective at times that what we need is to move beyond just looking at the technology to the person the actor behind that machine and that's what we're trying to do here all right I think we're running out of time so I'm going to stop here and hopefully take some questions if anybody has any questions how much 10 minutes or so we have about 10 minutes so if anybody has any questions yes yes I have a question cyber risk index is it peer-reviewed and is it being used in practice in the industry and so what are you saying right so this data is all coming from companies right now the index is still it's it's in review right now but the model that's using to come up with this is already peer-reviewed and it so it's available to the public the 40 questions is that available it's a public awesome the 40 questions are in that's in that instrument but we are refining it and as it gets better we're gonna just put it out there so people can use it okay all right um I have it my question is about empirical results with actual intrusions so is there any data coming back from these companies where instead of a simulated phishing attack it was an actual phishing attack and seeing how the CRI of the employees are lining up against where there's a a real intrusion as yet there isn't any to be honest so as yet the only way we're doing this is we're using the existing pen testing methodology and we're using our approach to give them the index and trying to see if the pen test data matches it but unfortunately we don't even that data is really hard to get I mean that seems to be the bigger problem and and to answer the peer review question I mean we're going through you know the bad guys out there are attacking us every minute and my peer review cycle is six months to a year so you know by the time I get this thing through the process I mean we're a year down the process the bad guys have already hit us like ten other times or 100 other times successfully yes um yeah sorry that just got kind of out of curiosity like I mean it's confusing why's 76 to 100 would not better for CRS Cortlandt is an example on this chart here what's with that green bar on the far right what's that is that little guy just like an outlier it was just there are outliers so the problem with working with people as you know is it's not we don't get perfect data I wish we could write I wish we could get that hard data right two molecules of hydrogen and one molecule of oxygen always makes water but people unfortunately it's not and so we're gonna have outliers on both sides so here you have a bunch of people with lowers lower scores on this side also not getting fished so it's not very clean and that's why the more data we get the better this gets but on average what we're finding is once you get beyond 75 these guys are very resilient okay once we get to that threshold and this is pretty compelling once you get below it we're in a whole world of trouble okay and that helps in itself yes you can add other types of security behavior like employees making choices about hand a handle customer data and stuff like that right and so one of the things we're doing is we're trying to predict what what I like to call cyber hygiene but within an organizational setting it's about handling PII more more you know hygienic lis for lack of a better word so we're working on it the issue is again we're still collecting data to make this much more robust than it is how much it applies across the board we still don't know because that's where the training comes in so if you can clean people better and train them based on some of this at least you have a metric that you can fall back against okay yes I just want to point out something that it looks like you discovered inadvertently that's really cool you were collecting information on why people click the link after they did it and you found some interesting information some people were suspicious of it before they clicked it but it seemed like they just didn't know what to do with that link so I'm thinking that's something that companies could do to be more resilient is they can tell the employees what to do if they're suspicious because they might not know like who to go to or who to send it to and then instead of all this negative reinforcement like you've got to embarrass or punish them you can start doing positive reinforcement like rewarding or recognizing people that discovered the vulnerability or the attack before it got widespread right and that's a great point right so we got to come up with better mechanisms of reinforcing people there's a whole part that I didn't talk about for instance you know we have tried different approaches to improve behavior a lot of behavioral training today misses habits right what are people's routines and so one of the things that we've been working on is how do I make you develop better routines and part of that routine might be you know making you more conscious of using to affair as often as possible or multi-factor authentication of looking at your logs of emailing you know problematic emails that you see to somebody in IT assuming that there is that setup for doing that and what we find is you know some people benefit from that but not all right so you need to be able to say okay who benefits the most from an habbit intervention and so again using our measure we were able to pinpoint where that CRI is coming from and then go to that person and train just them and we're actually doing that I have some data on that and we find that you know within three to four weeks you start changing the way these people are thinking about their routines and one of the actions that we did is we took away their mobile access to email so we cannot right off the bat said these people shouldn't have mobile access because their habits are really problematic and I'll give you some data that is really really interesting I can't explain it yet but there were people who click on links multiple times I don't know if you guys have seen this in your data set but we're starting to see this where people are repeatedly clicking on a phishing email link or repeatedly opening an attachment and it boggles our mind and one of the things we realized was a lot of the mobile clients that people are using are loves you to have the email come back at a later date so the person is clicking on and saying I'm going to come back to this and then they come back to it and they come back to it so you're getting multiple infections from that same person in theory and some of these people shouldn't have access to it by email by on their mobile device so we just took it away right so that's one way to think about it all right any other questions I'm interested that ye this study kind of depends on self-reported information how much data or is it ever possible to infer some of these behaviors particularly habitual behavior where you you have information on devices how are you cross-checking yes yes so one of the good things about using a lot of these training packages is depending on which one you're using to do your testing your pen testing for instance your red team's or whatever they're using you can say okay how are these people accessing the link are they accessing on a mobile device are they accessing on you know on a PC environment how many times you're clicking so some of the behavioral data we're correlating with the self-report data that we already have so we've been doing it in multiple ways we've been doing a pre post we'll be doing that post only so so what we're trying to bring this to is forty questions that we can use with regardless of whether you're getting tested or not so we can just use it across organizations I mean that's the standard we're trying to get to and and I think we're getting close all right any other questions yes my very subjective observation is that sometimes people click on links simply because they're curious right have you explored the role of curiosity and motivating people to open phishing emails well you know curiosity is more it's it's a very subjective notion but where we get to that is with our trial its expectation so more often than not I mean there are going to be those people out there we're going to click on a link just because you know I remember the Israeli Defense Forces were fished they fell for a phishing attack which basically said you know girls of the Israeli Defense Force it was a youtube link that was sent to all of them and that I would call curiosity but now if you fall for that in 2017 you've got a problem right so we get that 1% we're gonna click on pretty much anything right but we got to move beyond that to say okay what are the expectations hopefully we can protect against the vast majority of this but if we can find what those expectations are that people have and match it we have a better deal at stopping it sure yes did you I have attempt to make a correlation to like age group or our functional group within the company we did age doesn't predict much except at the upper ends of it it really doesn't between groups in the company in terms of what their functions are we find technical people fall just as well as non-technical people who are in info sack they think that they were smarter than they are okay we all do and one of the reasons for it and it's a very simple reason is there is just so much that we can do to match an attack to you that it's likely I can catch you if I really want to right if I'm really motivated I can do this and then what we find is repetition helps so even with highly technical people if I repeat that attack the same attack the likelihood of us getting twenty fifteen to twenty percent is not not pretty long we can easily get them so we're not seeing big differences in them okay hold on to that do you see more attacks to like the company executives in higher level on the spear phishing we haven't done only company company executives but you know the likelihood are is that you know if you're a company executives so what are the things that we've been talking about is how do we take this index and figure out who gets admin privileges like who gets more access to databases for instance and what we find is right now the system we have is based on your role so if you're the CEO you get access to things which you may or may not need access to but you force yourself into it you sound to see I can do this I need access to the accounting or the billing system maybe we don't but how do you say no the only way to say no is if you have some quantitative metric that says you know what you are too risky for us right now and maybe you shouldn't be getting access so that's kind of like where we're trying to take this and say we need a quantitative metric here of some sort that tells you know the IT guys out there to say wait a minute no let's let's come up with an index let's use this index and let's give access based on that thank you yes all right out of time thank you so much thanks for coming

Channel: Black Hat

Views: 24,431

Rating: 4.0213776 out of 5

Keywords: Black Hat, BHUSA, Information Security, InfoSec, Black Hat USA 2017, Black Hat 2017, Black Hat USA, cyber security training, cyber risk index

Id: 3L3IrAN30a4

Length: 49min 47sec (2987 seconds)

Published: Wed Nov 29 2017