Artificial Intelligence (AI) in risk management - IBM, 4th-IR, Microsoft & Google

good evening my name is BAM hat Burnie Yanni Chomsky Managing Partner of trestle group the sponsor of tonight's event mean my colleagues see Shan and Tom are extremely excited to welcome all of you to tonight's discussion on artificial intelligence in risk management as financial advisors in the banking industry we believe that the way we are doing business in banks will change digital transformation will not only impact human behavior but also create new opportunities for the way we are conducting business artificial augmented intelligence will enable innovations that make banking processes more efficient more effective and less risky and it will enable decision-makers to embrace upon new challenges a new business model they will lead us in the banking future as consultants we believe in empowering innovators in creating the future of financial industry we are extremely excited and looking forward to tonight's discussion and with that I can introduce to you tonight's panel grace praising ttan from IBM Frank Leutz from fourth IR Martine Miller from Microsoft and Mack McNeil from Google so let me try and tell you a little bit about how this might work in a space where you have old and new technologies so think about a world in which we only want to distinguish between borrowers based on two attributes FICO score which is like a credit score and income and let's assume we have an old technology which is like a linear technology okay so in this world I might say anybody who has FICO and income above this particular line over here is a good credit and anyone who has FICO and income below this line is a poor credit and now think about a new world in which we have access to this fancy new nonlinear technology which has this shape here where the decision boundary is actually in this kind of L shape okay so this is like a machine learning scoring method for example and what happens here is that you know now suddenly all of these people sort of look you know kind of less risky under the new technology than under the old technology but there's a group of people who lose out by the application of the new technology and you can start to see this here okay so when the new technology comes in and we have access to this new machine learning method maybe suddenly our black box algorithm tells us that the truly risky people are sort of all below these blue lines here and the less risky people are above these lines here and so what happens is you're going to clearly have some new people who are winners but some people who are losers when the new technology comes in because obviously here these losers used to be considered less risky under the old technology but are below the blue lines and are considered more risky under the new technology why is this interesting because if we think about you know minority groups and majority groups in populations say we divide them by race or by gender or by things like that and we find for example that the minority group is sort of is sitting over here and has FICO scores and income levels that are around here whereas the majority group is over here what happens when you introduce this new technology is it looks like minorities lose out and majorities win in terms of their credit scores okay so this is kind of an interesting aspect and I think more and more people are starting to think about this where we have these unusual social science aspects that we need to think about distributional implications of new technology risks that we might not have thought about before that suddenly become apparent and so this is the sort of thing that we need to be doing more of which is thinking about the way that society is going to evolve as a result of our new technologies it may be that we have more efficient risk prediction tools at our disposal but suddenly these new technologies are opening up potentially more indifferent risks so I'd like to encourage you in today's presentations which are terrific of course and panel discussions to sort of expand these definitions and think really carefully about how AI is going to affect our society and to think about the risks that might be involved thank you everybody for coming to this evening thank you for Imperial College and trust a group for co-hosting this event and help organize everything the Institute of Risk Management is the leading body for advancing the profession of Risk Management we are involved in qualifications that are going to be recognized we do research we also set standards for competencies and risk management and we have recently in fact as recent as last week just launched our new qualification of International certificate of risk management and financial services so you would have seen some information around that about that and around and we also have what we call regional interest groups and special interest groups and this particular event has been organized by our banking wellactually enterprise risk management in financial services and banking special interest group and we have been quite active we won't quarterly events we've have had previous sessions on FinTech risk appetite and I must say this is very very well attended tonight in fact I think it's one of our biggest attended events and so what I'm going to do is I'm not going to go on for too long I'm going to introduce you to the chair of our financial services sig Raisa Sadiq who are then going to introduce the speakers thank you very much and enjoy the event we are very honored to have with us basically grace braising ttan from IBM who's flown in from Jacksonville Florida Frank looks from forth IR and Martin Muller from Mart Microsoft and of course Matt McNeil from Google what an amazing panel I couldn't have asked for anything more my focus today is all about artificial intelligence and how it impacts risk management as a profession how it impacts industrial sectors be it financial services or the healthcare it doesn't matter but the industry why it's a much bigger question and I think the questions hopefully that I get today from the audience or we get from the audience will be of that nature so I'll feel the questions I hope that works for everybody yes I'm gonna say except that it's a yes and now we'll move on to introducing grace braising ttan who is from IBM thank you so much grace so let's get started how many of you have knowledge of artificial intelligence machine learning or cognitive systems raise your hand good number so there's a the first few slides I'm going to go through very quickly because this audience understands that many of you already understand it but but I like to start with this slide because the reality of today is that companies are facing disruption and they come to us and they come to the other technology companies and they say I'm drowning in the data I have too much of the data how can I use information more effectively and and one of the key things they're saying more and more now is an app is disintermediating me and if we think about Airbnb and we think of these technology companies that have come into play and how they are disintermediating us from an industry perspective disruption is an opportunity it can be an opportunity for those that use information effectively and recently we had a top for automotive CEO in our facility in New York and we asked him what his biggest concern was and he said from a competition perspective it isn't the other car manufacturers it's uber because his concern is in large cities people don't want to own a car and nowadays we have to think less about the driving experience and more about the passenger experience how can we give the passengers the best experience at the lowest costs and that is the nature of what we're dealing with in terms of this industry today it's all about immediacy it's all about click I want my car I want it now in banking I've been in banking for a long time most of my professional career and I remember the days when people said oh it's all gonna be digital there's not gonna be a need for branch anymore that's not true it's all about the ATM it's all about the branch it's all about the call center and it's about digital we want it however we want it whenever we want it and we should get it that way and this is the environment that we're in so you know one of the examples from a risk management standpoint or what we're seeing at IBM and in other companies is this need and I look at the private wealth space the affluent market what do I want in my investment advisor I want someone who knows me I want to be able to opt in and have them take a look at what's going on with me from a social media standpoint I want to be linked with them I want them to know what's going on in my life why because the more they know about me the better advice they can give me my daughter's about to graduate from college theoretically she'll be off the payroll in April but who knows I've got another one who's a sophomore but in any event he knows how to talk to me I want him to advise me I want him to use the information and to see from a risk perspective what risks I want to take on so that I can do business that way I want to know what others are doing I want to know what other returns are but at the end of the day I'm gonna pick and that is the essence of where we are from an overall industry standpoint now I was in banking I was a former chief compliance officer I joined IBM about 18 months ago and one of the issues I had income and risk management was data I had data I we were awash with data 90% of the world's data was created in the last two years 90% of the world's data was created in the last two years 80% of it is unstructured dark data it's the voice it's the email it's the text it's the blogs and 70% is the rate of increase that we're seeing in financial services every single year 70% increase in data so the issue isn't data its how do we take the data and translate into information that we can use from a competitive advantage perspective how can we use the data to do a better job from a risk management perspective so when we look at the environment that we're in today we've got pervasive interconnectivity I mean look at the power of our cell phones that's a power of a computer years ago we're always connected all the time I do a lot of travel I see people on their phones all the time we have an explosion of data by 2025 180 zettabytes of data and our view is data is the source of competitive advantage the more you know about your customers the more you can transact with them the better the situation is so data is the natural reservoir that we have and being able to use that data more effectively and more efficiently by using some of the artificial intelligence or cognitive technologies is key now when I joined IBM 18 months ago I have two girls they didn't really know what I did she's some vice-president she goes to she has a lot of conference calls she goes to meetings she's traveling a lot but then 60 minutes did an expose about Watson and what we're doing in the cognitive space and in that broadcast we talked about what how we started from a Watson perspective and what we did was we ingested research papers medical research papers and we leveraged that to transform how we're doing cancer treatments today from an oncology standpoint we took the best of the best in Memorial sloan-kettering and we took the MRI images and the x-rays and fed it into the system and we took all these research pay and diagnostic reports and we said okay let's see how we do and oncology is a tough business and the Memorial sloan-kettering oncologist said yeah I don't think so skeptical I don't blame them but the reality was when we had done that and they looked at cases that Watson identified and their best of the best oncologists we were spot on in terms of the approach and the diagnosis where we were different in 30% of the cases was in treatment options because Watson identified new research that the best of the best of oncologists and Memorial sloan-kettering missed why because there's so much data out there and so to the extent we can use that data more effectively that is the ultimate value proposition so I look at it I say well there's machine learning and statistical analysis there's artificial intelligence and then there's cognitive and we do believe from a world perspective that every decision that mankind makes in the future is going to be informed by some cognitive system or technology and that the world is going to be better for it think about what we were able to do with Memorial Sloan Kettering now what if we could apply that same type of technology to third-world countries who don't have a chance at finding a good oncologist we're fortunate you're in London I'm in the United States we've got some great technology capabilities the best of the best in terms of hospitals but other places don't have it so these systems have the ability to understand to reason and to learn and while humans excel at imagination and they have the ethics and the compassion cognitive systems have the ability to locate the knowledge and the information and to use the natural language capability to be able to inform us what are we doing give me some practical use cases grace talked about what we were doing in the private left side but in risk and compliance we're seeing these technologies being used as it relates to AML kyc know your customer the more I know my customer the more business I can do with them and oh by the way I have to from a risk management standpoint using it for conduct risk purposes and surveillance technology in we're using it in the regulatory compliance space to be able to merge between risk and compliance in the control infrastructure and the new regs and policies and procedures these are real capabilities that our clients are enjoying today and at the end of the day clients data we basically protect IBM is a company that trans acts with other companies but we will touch a million I'm sorry a billion consumers by the end of the year in terms of Watson through the business customers that we have relationships with so our view is clients have the choice and control over their data if you're gonna use your data to train these systems it has to be yours that's your competitive differentiation that your natural reservoir and we're going to use domain expertise in this process so who are training these systems should be a question that everybody should be asking how am I getting this information who's doing the training and this is where from an IBM standpoint we've made acquisitions in the risk management space because we said we gotta merge what's going on from a regulatory perspective we've gotta merge in terms of what's happening from a client perspective and we've got to take the technology and integrate those three things together because one by itself isn't gonna get us to where we need to go from an overall industry standpoint we think this is transformational I think you do too because you wouldn't be here your time is precious too and we do believe that there is going to be a new paradigm well it's a new color jobs so before we had blue collar white collar new collar where you don't have to have a four-year degree but maybe a two-year degree where we can transform what we're doing from risk administration and the manual processes that exist today to true risk management I want to end with this case study I had the privilege to sit in on some of the engineers meetings and we talked about the importance of working together and we have established a partnership on AI best practices with the best of the best IBM we can't do it all by ourselves we're gonna partner with the Microsoft with the Amazons and the googles of the world because at the end of the day we have to do things in a way that's ethical all the technology in the world the best man and machine woman and machine it is together it is all about augmenting intelligence not displacing it so with that I'll turn it over to my next speaker Frank thank you very much I must say I feel like a kid in the candy store or maybe a lego lamp or something like that you know somewhere in 1976 I started playing around with this stuff and I better push on the button right okay here we go somewhere in 1976 I started to play around with artificial intelligence I mean many of you in the room were probably not born by then but yes we had computers and we were able to do something with that and ever since I've been working on this stuff and today now I mean maybe for a couple of years now all of a sudden it's all available it's out there you can start working with it and when you can start working with it and start combining it with the use cases where you start seeing what humanity needs then you get really really enthusiastic about it and that's why we formed the fort I our company fort I our fourth Industrial Revolution so with a bunch of guys we decided that we really need to do is take that tremendous opportunity of having all these components available whether it's IBM whether it's Microsoft whether it's Google I mean I'm probably another 500 others they get smaller as you go down the list obviously you know you can bring it all together here you can start building things for this so the time is now I'm excited about it my partners are excited about it for they are excited about it we believe that you can leverage this emerging science of artificial intelligence cognitive computing call it machine intelligence whatever name you give a deep learning you can leverage it to make the world a safer more sustainable but also a more productive place and we can do that by building practical solutions that are easy to understand and simple to use so we don't want to argue with a complicated complex solution we want to make sure we deliver something that is understandable so the time has come and I would suggest that you invest now to be a leader or as far as I'm concerned you can also pay later to be a follower but that could be a little bit more expensive so warn you for that now I hear a lot of concern about wow human intelligence will computers ever replace humans etc well look at on this screen here that's humans for you and I'm not going to say that humans are flawed in general but you know Deepwater Horizon to deep horizon what do you know the rig out there in the Gulf somebody said yeah I know but I'm gonna take the risk well thank you very much you know you took a risk you know think about Germanwings some you know you think that if and I'm sorry I'm gonna say it if I if you would have yours used IBM Watson personality insights and would have tracked every email from every pilot there wouldn't have been a red flag that this guy was a little bit emotional and stable so that isn't avoidable personally I worked on a project a couple of years ago on healthcare acquired infections in hospitals that's a bit of a controversial topic but roughly between 200 and 350 people die every day in the United States alone of infections they acquire in the hospital so applying artificial intelligence to approve that if you only improve about 10% you save 25 lives a day I'm just going to take the low number so yes humans are somewhat flawed and there is opportunity however at fort IR we do believe it's the symbiotic combination of human intelligence and artificial intelligence that is going to bring the breakthrough it's not one replacing the other now this is a human credit goes to my partner Thomas here he's a bit obsessed with Homer so he insisted that his slides get in here I'm not going to go why but so we believe you look at this that systems machines whatever you want to call them can actually see better today they can hear better they can definitely read better I mean grace just told you the thing can read millions of documents in the blink of an eye it gets certainly reason almost as good as a human if you put all that technology together making sense out of that all that input it's pretty darn good when you start working with it and you know we did depict the brain a little bit smaller of this artificial intelligence system but it's all there now what the human brings to that and it's a controversial thing is the human biases I mean a lot of financial people in here and I know know if you do this but I normally when I make profit in the stock market I sell too fast and when I'm losing I'd never sell until you know I really have to sell right so that's a clear cognitive bias there are some books out there described 120 different Caucasus biases that people have one of them I would recommend is you're not as smart as you think you are it's a very interesting book to read or another book it's you know predictably irrational that means we're very predictable although we're very irrational good book to read and then of course the invisible gorilla is always a classic right where you know there's so many things we don't see so you got to deal with that as you deal with people so let's move on wanted to talk about some of the examples now our company fort IR has been working on health care so far so the tree thinks we've really delivered on so far our biopsies you know understanding biopsy slides interpreting them and we scored pretty nicely there I mean we came in 15th on the list of 450 participants and I may say I don't didn't see Imperial College on that list but there was some the notaries like MIT and Harvard and Heidelberg and AC Anderson on there and they were in our neighborhood so we're quite proud of that as I said I worked on the healthcare acquired infections which we were able to identify some very hot spots in the hospital where infections were being hosted and we're now just delivered a system for open-heart surgery readmission and those are very successful they all save patients lives and they actually save money so that's always a very important part of it now what about risk management I think I've heard grace say it already data it's gross it is dark it is incomplete it's ambiguous it's a mass but don't worry because if you don't like yesterday's data you're going to have more tomorrow so if you start working on making it better today then you will have better data tomorrow to work with what would I do if I were to be a risk management I would start collecting in a big data Lake Martin can help you with that you know perfect place to put it you know apply probabilistic and deterministic and analytics to it don't forget the old stuff but start applying the new stuff cognitive computing natural language processing all or gate kind of machine learning but then also use tools to link it back to the youth to the human being here's an example of such a tool Panama papers go to the website interesting thing to browse through and that's it so Ford I are we do believe that the symbiotic relationship between human intelligence and artificial intelligence can lead to great things thank you very much I'm gonna introduce Martin Mulder from Microsoft after Martin it'll be Matt McNeal from Google all right guys what is Microsoft doing in the AI space it's a challenge I'll try to rise to it first and foremost artificial intelligence actually we do call it slightly different it's augmented intelligence yeah when I enter Microsoft yeah great it's funny finger PR it isn't actually I was very impressed when I came in and it wasn't so long ago to find out that anything and everything we do around AR is cluster around those four ones I don't have a Homer Simpson but it would have worked as well here so the but see speak hear and understand and actually this is truly by the augmentation so if you need another pair of hands how do you get it this is it if you need a hundred million hands if you need a hundred million eyes this is the idea behind it it's not about competing with Hugh it's not about replacing humans it's about the augmentation of it because today we have so many tasks we have so much to do and we heard a lot of different areas already from Frank where we can massively improve we do need these augmentation to master today's complexity of course these are very interesting words it it looks nice but it's a bit of abstract thing so I won't take at least a couple of minutes to go into what do we really understand this and give a couple of examples when it comes to seeing Microsoft doesn't build any self-driving cars yep but we do of course work with all the major car companies in the world in terms of helping them get there and seeing is an absolutely essential thing there is this occasion running across the street or is it just a plastic bag being blown across the street there's not a mile of distinctions that show major one and of course later when we talk about the ethics of AI when you have an AR deciding should the AI also be held accountable if you have to brake to avoid hitting the kid but that might actually were the 90% probability in danger the five people in the car what do you do these are all the key things that go to it and without seeing you can't decide now speaking and hearing actually we have another company event this year it was in loveless Vegas so we had good fun but we did a bit of work and being old and boring I had no idea who this guy is apparently it's a big pop stone Asia and they invited him on stage unit in live rap battle Ryan battler Harvey on the quarter was oh one of ia eyes it was actually very very fun to watch him rapping in the microphone she coming back it was very good entertainment but it's not just good fun well we really want to take augmented intelligence seriously we need to be able to actually interact with machines that we've heard it a couple of times natural language processing is abstract as it seems it's absolutely central machines need to be able to hear us we need to be able to hear them so they need to be able to speak back it's a key thing if you do want to take augmented rather than artificial intelligence seriously now the last one the understanding one this is one of my favorite examples really we're all in banking here so obviously it's about how can we make more money how can we have less cost and PTI goes up but this is one of the examples where AI actually has a real impact on people's life and making it better and the lady with the red hair she's a actually I think she's based out of London she's a creative director she's a designer and unfortunately when she was around 29 she got Parkinson's now I didn't know really much about Parkinson's but read myself into this case of it ultimately it goes back to your brain sending out signals of a signals over signals that basically are becoming a vicious circle and they overload your nervous system and that's how you get these these tremors these Parkinson's tremors it can imagine if you are a designer and you suddenly can't write in name anymore that's while the dramatic and the lady on the right-hand side she's based out of our AI research lab in Cambridge and she came across this as well and she started tinkering a bit she said look if we can actually understand how these kind of brain waves these kind of interactions come to be can we know then also change this and she was sitting tinkering and if you go in just google the markers of a memorial to see videos of it sitting in a London apartment building what eventually became the EMA watch and what a theory is actually it analyzes these different brain words these patterns in the brain and you wear this watch it emits a pulse into your hand and ultimately shorts cut short cuts this vicious circle of the brain waves overloading the nervous system and you can see what happened from the left so top there that was Emma actually writing something without the watch all on the right hand side you can see it with the Emma watch on and for me there's just a fantastic example of a is not just about funds and banking as much as we all bankers here it's about something that can really really help people now we are after all there in an AI in risk management event today so pattern recognition is something that then also goes into machine only for finance at risk one of the key things I think we all know about is stopping prompt fraud is a massive problem for everything in finance industry from identity theft to you name it and here it goes on to what are the patterns of usual behavior how do people usually interact goes on to also misconduct what do they usually trade do their trade low wages high values we all can detect these patterns in excel that's no problem we can do this but if you actually have to do that scale around millions hundred millions of data points that is very very tricky adhere pattern recognition machine learning for area for risk that comes into into its area we're working with customers not only detecting more all than before but actually taking it down from minutes into seconds of detecting it another thing is of course biometric security I think this morning I read in the Financial Times that the SEC in Europe was actually hacked so security is a key key element of our digital future for us personally but also for institutions security making your own institution's cybersecurity a hundred percent safe is key to managing risk you'll never be a hundred percent but where we are now it's not where we should be and this is another easy example of actually the way we use a trackpad or a computer is just as unique as is our fingerprint and these are just extra examples of people using trackpads on on their laptops and I can also hear detector patterns how does somebody usually interact with their device and suddenly if somebody is a bot trying to do an attack on your computer on your system they will be completely different if I use the scroll pad on your computer I will be different from you so here again cybersecurity and these are very simple solutions make a massive difference when it comes to managing the risk in your institution last but not least we need to look into the future and they're working with a little bit insurances in this world we just had the hurricane season and grace can actually tell you the first time tell being from Florida in the u.s. so here a massive amount of data you cannot imagine the huge amount of data that goes into these kind of systems data Lake we already heard it and how do you get all this data how do you put it together garbage in garbage out you actually need to really work with their data also to then find out when it's the likelihood the hiset something is going to happen not just to save lives but also again we're in risk management to manage this to manage the impact from the insurance industry to the insurance industry to how do we rebuild the country or how do we rebuild where to reco later on going into the finance industry and those are just a couple of examples and I don't actually see who's supposed to give me the time up there two minutes fantastic I can give time back because I managed to then to give you a brief overview of what Microsoft is doing in the AI machine learning space right now in eight minutes I'm very happy I hope you halfway enjoyed it and I'm looking forward to having a really good debate later on when we go into the Q&A and now mass from Google well welcome everyone and I'm Matt McNeil I think I've got one of the best jobs in Google and my ten minute introduction is just to give you an explanation of why I think that's the case um I work in a part of Google which really represents Google's infrastructure and really where does that infrastructure come from is an interesting question so 19 years ago last week the simplest webpage on a web launched and between now and then in order to drive its page an immense amount of R&D has been executed over the last 19 years so to serve you searches and cat videos or whatever else you can see from Google is actually some of the world's largest infrastructure this is a picture of one of our datacenter halls it's actually one of many on that particular campus that campus forms one of a cluster of about three campuses which population groups about half a billion people when you do a search you're searching an index of about a hundred petabytes that sort of order and getting a result back another 200 milliseconds and it ought to give you the best result possible machine learning is operating within that window at that sort of scale now that's for stoves over the last few years actually the last decades to really drive thinking about how we work with data something gray said earlier data is the rocket fuel which you need to be able to start things like machine learning and this is a picture of some of our latest racks actually and if you want to geek out for him these are what we call well they're actually custom chips we call them tensor processing units and for those people in machine learning we sort of outsource probably or open sourced a few months ago or years ago and something called tensorflow which is how we do machine learning at scale internally these racks are between sort of eleven probably those ones on the picture they're between 11 and 12 patter flops of processing power that's sort of supercomputer range and they're full of custom chips purely to process machine learning now that's quite a lot of investment we invest probably about 10 billion a year in some of this infrastructure and behind this as I mentioned we have this track record of publishing open papers so just I've just picked a few out here on a data processing side to illustrate this some of the papers were publishing 15 years ago which led to some of the core tools you'll find in a big data space like Hadoop which allow you to start working with data what my job is and why it's interesting is I get to work with the bowels of Google's infrastructure and R&D in this geekery and about five years ago I was in a room with about 20 or 30 other people debating whether we should make these available to people fortunately we did and we now have four or five thousand people working on this so all of those are now available and what my job it is is to take them out to people and figure out what we can do with them because here's the thing we don't always know we know how it works in our environment we don't notice they know what innovation is going to come out by deploying it into your environment so that's the fun I get to have working with people at HSBC for example in the finance industry at the moment now machine learning is a core thing it's probably one of our number-one priorities at the moment and that's simply because we've got to a stage we believe technically where we can start doing machine learning at a scale which makes it really feasible machine learning and the techniques that we're using aren't particularly new they've been in the technology space if you've done degrees in computer science probably for two three or four decades the whole area of artificial intelligence by the way is much bigger than just machine learning but machine learning is where I think a lot of the advances have been made recently and this is our CEO sundar pichai and when he talks about this about 18 months ago our entire company pivoted and so when you look at our private product stats this is a pull from some of our source code directories of how many products we're launching today which have machine learning as a core part of what they're doing so our translation systems for example move recently on to machine learning platforms when you're using your Android phone and to ask your questions or Google home you're using machine learning model and some of the more interesting areas is what we're doing with deep mind how many people here have heard of alpha-alpha go in the so what really staggered me was this fact here the alphago board 1999 team board has 10 to the power of 172 combinations now that sounds like a big number but just how big is that the number of atoms in the universe is about 10 to the power of 80 so what it says is this is a casa problem which unlike chess which can be brute forced you simply cannot brute force this sort of game so why we picked it we wanted to see whether we could get something which could recognize the aesthetics and the patterns within a board like this in order to win because you simply can't calculate the best outcome you've got to feel it and so these technologies are starting to make really the techniques it's starting to make technology in a way more humane rather than just rational now that does lead to a lot of ethical questions a lot of societal question which I hope we're going to dig into a little bit on the panel later we also been doing some very interesting learning on the cutting edge this is another piece of work by deep mind and it's where we're starting to move from what we call supervised learning into unsupervised learning so what happens when you start building deep neural networks or machine learning systems where you're not giving it the answers to train on you're just giving it the data so in this example we were building systems without telling them how to walk or show them how to walk seeing if they could figure out themselves how to walk so self learning and this is a key area as we move into unsupervised learning territory which becomes very very interesting because of some of the biases that you sometimes get within the data centers that we bring perspectives to to the data and to the answers that machine learning algorithms can find and we started applying this internally let's make it concrete we applied a machine learning algorithm to our data center operations we've got a lot of smart people in our day center they didn't think we'd be able to find something that more optimal than they were already figured out over the last 10 years 40% reduction and by the way we spend a lot on data center cooling and so this translates to you know how she was a multi millions of dollars every month in savings and what you start seeing is here's a multivariate problem hundreds of different variables being optimized not maybe once a week by some expert at human beings but once every 30 seconds by something's even more expert and so you can start doing things you really couldn't be imagined before and on the other end of the spectrum they're getting applied to our system so for example inbox one of our email systems we launched this Auto reply so a bit like spam filtering checks your email for spam to get rid of what this was doing is saying this email looks like it's gonna generate a response actually we can give you the top three likely responses and what we found is after launched with within within probably a couple of months ten percent of email responses using the system we're using also replies and for those of you that are managers this is exactly the sort of thing no one could tell the difference and it works really well and so the way we kind of been thinking about this and externalizing these systems position I mean machine learning site by the way in Google machine learning is just part of our infrastructure all of that work is done within our infrastructure teams so we have this concept of infrastructure that goes very deep but um as I've mentioned earlier we open source through tensorflow our machine learning systems and they're available you can run them on your laptop you want to but what we've also done is started looking at all the models we've already trained so take google photos with the vision API or when you do an image search on Google that's a machine learning problem understanding what's in that image being able to identify it when you do translation recognize speech analyzing text sentiment analysis all of these attractable via machine learning now we already have some X of models in that space so on one level we've just been making those directly available as part of this offering and that's been driving some really interesting work from some of our customers that we're working with and here are a couple of examples this was a very small Providence called Kiribati and they control actually quite a bit of sea but they don't necessarily DP as you'll see in the problem and what they were finding was that their seas were being overfished and abused they couldn't track who was legally transiting the ocean and who was doing a bit of cheeky fishing on the side so by applying machine learning you can tell this is a trawler this has got these sorts of nets and they started making fines which accounted almost for 1 percent of their GDP which is quite quite fun and buy or sell Descartes labs studied forecasting us corn production they got it in under 2% and in terms of the accuracy there it abled enabled them predict ahead five months ahead of what the official results were I say when you're a corn trader which is what they are that's pretty big a competitive advantage and which was fascinating on the insurance side and we're doing a bunch of work with AXA you know I think no point 1% of your cases generally produce sort of returns or I should say payments over over 10k identifying those is obviously critical when you think about the volumes of cases and we were using the tensor flow models with these guys and and started get an 80% prediction rate accuracy of drivers who are likely to be making those sorts of claims or whether those claims would payout and huge reductions in cost that came from that one of the more interesting ones we did was a with the USAA so these guys we've been part of with them taking the image api's that we have and allowing them to extend it to be very very specific to their domain and use case in this case being able to take a picture of a car and recognize which bits of the car are likely to be damaged and how badly damaged so that as an Assessor you just take a few pictures almost in real time you can start getting assessments back as to whether this is the likely payout or not and this maybe cost them a couple of hundred dollars a month to run and but maybe a million times that in terms of operational savings that's the sort of examples of problems that you can solve with machine learning which I think can really change the game for a lot of companies and for us had a nice picture here of Harry page which obviously I forgot to pay Steve and so I'll skip that one what area and so Larry Page made this statement with Charlie Rose a couple of years ago which I think really encapsulate the way we think about this massive change in technology that we're facing today we are in a generator of shift of technology paradigms probably something we haven't seen for 30 or 40 years the opportunity is huge what we worry about almost all the time is are we going to miss the future and I think these days everyone's a technology company this is a bit of advice every single one of us can take to heart so with that I think I'm going to stop and maybe rather we'll move on to the panel thank you into our first question to everybody are you ready how risk-management do you think will be transformed through artificial intelligence question to all of you start first I'll start first so we look at risk management and and we see opportunities in multiple ways again I talked about it moving from risk administration to risk management and one of the things that we have focused on from an investment perspective is taking some of our Watson capabilities and incorporating it in the first area and you alluded to it is in the know your customer area to the extent we can improve that client on board process that's beneficial from an overall industry perspective and using some of the cognitive and the machine learning capabilities to onboard clients more efficiently and more effectively but also to make sure that we're assigning the right risk rating associated with those clients because if you get the risk rating wrong it feeds into your transaction monitoring system and it just generates a bunch of alerts the second area is in the AML space in terms of the alert monitoring in the triage process because again ninety-eight percent of what we're seeing from an alert perspective is false positives so again to the extent we can use some of the cognitive capabilities and look at you know prior history prior work that had been done that's a huge value and then generating the dossiers associated with what would be filed from a SAR perspective again that's an area focus so AML KYC you had talked about what what what's going on from a fraud standpoint and payments we're very focused on that but the other area is around surveillance insight and and the reality is is that regulators are concerned about market effects and what is happening from a control infrastructure standpoint as it relates to monitoring activities and so one of the things that we've really focused on at IBM is is basically looking at voice data electronic communicate Asians in terms of email and using some of our API is associated with that along with chat rooms and in my prior life when I was the CCO whenever I had concerns around a particular employee let's say they downloaded information too much information too much NPI red flag then all of a sudden we would launch an investigation how can you bring the data together more effectively and that's where we're going from an overall surveillance and conduct risk standpoint that we can identify anomalies in advance so we can look at the HR data and say yes this person didn't get a bonus all of a sudden their email correspondence and again through api's went from a happy person to an angry person and all of a sudden in chat rooms are complaining about their boss and leverage that information not to make the decisions because it is about augmenting intelligence it's not about displacing it but using that data and providing that to the compliance professionals in the first line in the second line of defense to say here's the information that we have so that they can decision whether it's an issue or not an issue and then in the last area around regulatory compliance this is a huge area because depending on what research paper you're looking at we're seeing between twenty to forty thousand regulatory changes that are occurring in a given year there's no way any compliance organization can keep up with that amount of change and so what we see as an opportunity is merging what happens in the reg change space the twenty to forty thousand regulatory changes and three hundred million pages of regulatory documentation by the end of 2020 policies and procedures and controls and we're incorporating our cognitive capabilities as part of the GRC infrastructure so that where I've got my policies and procedures and my controls and my walkthroughs and my testing and my assessment process merging with the Reg change so that at some point in the future we can look at the real-time monitoring of what's going on from overall compliance perspective because we have to get to a point where we've got sustainability and compliance and we've got to move from the swivel chair effect that we see in AML kyc where we're looking at system system system swivel chair associated with surveillance swivel chair in terms of rank change and really use these technologies to inform risk managers more effectively would you not say that by using swivel swivel chair at solution if you don't you get whiplash absolutely do the swivel chair you're clearly your conversation or the narrative will be very clunky as well so you're not able to be your more known as a police function as opposed to a part of the business required function exactly well it's part of the business they're actually questions with who here is a actively working in risk management well said who of those that keep your hands up there's still bit of work out who are you guys things that you're the best friends of your frontline organization who thinks they'll want to be probably know yeah I love you all of course but and I think that's gonna be one of the biggest changes here what you can be the best guys you are but if the organization doesn't actually work with you if they see you as this police guy if they work against you you're not going to get anywhere you're always going to have to run after them you're going to get crappy data because they think oh no and yet another tool to fill out I just do it five minutes you don't get the right a to all of these kind of things and here actually when you come to having all the technology in your company with compliance and risk management built in that's what I think AR and Emma comes to their own and these kind of things so imagine somebody who goes to customer meeting and says well I'm going to go to a customer meeting put in my Outlook and say I'm going to talk to somebody in Germany about alternative finance well yep there is a couple of regulation attached to that usually they would have to go to your whatever portal is find the regulation find out oh yes to reverse solicitation that is something if this comes automatically wouldn't that be wonderful if things are being actually automatically prefilled if you actually become the if your frontline organization you don't have to run after the data it actually becomes something that is more or less convenient for them to do that's their helping this is where the tools come in where they can get a lot of stuff done for you you have to do less the organization has to do less and actually you don't have to hide from the near wall when you see them in cafeteria I think that's gonna be what the biggest change is coming but Martin my view on that is you know when I was a CCO or CRO I wasn't gonna be their best friend but they needed to know I was watching and these types of technology capabilities enable you to let them know if they are watching because then they come clean quicker absolutely they do I mean that's the reality and and the other thing I would say is it just isn't about technology it also has to be about the people that are training the system so you know about when I first joined IBM 18 months ago we did not have subject-matter resources at scale that we needed to train these systems and that was one of the reasons that we acquired promontory 600 people that had regulatory compliance background and practitioners because if you are only looking at it from a technology perspective and and the customer the customers don't have any patience I mean I didn't have any patience just put the fire out and move on to the next fire we are in that professional juggling situation right now so leveraging people the practitioners that have the experience that used to be former regulators with customers and technology company I think is is is the way we've got to do this because it can't just be about technology so it is technology and translating that technology to ask the right questions domain expertise be aware of what's going on before it happens totally compliant at least look at some patterns and the other thing is both of you loaded to is you can't be their friend but at least by asking the right questions they realize that they need to be your friend they need to come clean a bit more trusting person you can tell I would like to jump in and take a little bit of a different position I mean it's a bit of my role on the panel I think I have right and it's not the contrary to that but whenever you have a new tool and a new capability in this in this case artificial intelligence this also gives you new possibilities so you you're gonna have the ability to assume new risks and you also have the responsibility understands your customers and the technology they use so that you can't judge that risk much better let me give you an example yeah and it's it's it's a classic example I know when a jet fighter goes up particular an f-22 or an f-35 that thing is not flown by the pilot the pilot is along for the ride he gives the general directions but the the artificial intelligence in that machine flies that airplane and you could never fly that without the with the algorithms and I would say that in the future you're gonna have companies that can go on the business edge on the business you know take that business envelope to the edge because they have the ability to predict they have to the ability to deeper understanding you have to understand that as a risk manager so that he can facilitate that facilitation yeah well absolutely which is very similar to what everybody's saying which is exactly what you is the narrative you've got to change the narrative so that apps or loans as opposed to becomes clunky and swivels anything else you want to have I mean I think I agree very much with them the comments have been made around the sort of the processes that we really face today with regards to what we're managing risks for yeah and but I was reading that question a slightly differently or if there's another way of reading the question is what risks will we have to manage when AI is is democratized that's an interesting question um you know when you think that anyone that you might be worried about could themselves be using AI because it doesn't take a lot sorry later on our territory sort of thing well no I think that you know with any new technology there's no magic bullets really within this there's kind of a question because a really significant check no technology changes the environment itself absolutely and you know what we're talking about when we're looking at risk management is actually understanding the nature of the changes not just in its micro sense and the individual processes but in the in the macro sense of what risks we'll be looking at and facing and and you know we mentioned that the organization that I feel many of us on the panel here are subscribing to is a recognition of the fact that these technologies do have significant impact which are relative you know related to absolutely our social values yes and what we believe to be acceptable and and how they should be be thought unregulated potentially so I think that's where the questions come in and which is sort of the next derivative really of this broader question and I think we're understanding these technology what they're going to be doing will be a process of exploration to engage so interesting so what you're saying sorry if I don't it's what I understand this a little better so what you're thinking is potentially we have respect stirs and a lot of financial companies we are used to risk registers so what you're saying is possible potentially a clustering a lot more clustering of risks around more key risks which are combination of very other risks but I'm saying that there could be risk to our on the register right completely you know because we have we don't even know what their entire categories of understanding what the interaction of these technologies with society is sure the risks or with people's behavior and and as I say these technologies are not going to be something that is limited to large companies with high capex you know the whole of the technology revolution has been around a key thing which is the fact that 20 years ago the ability to deploy capital was a competitive advantage that kept people out of your bit out of your business areas and fortunately financial regulatory which helps maintains the competitive advantage to keep Ziggler notwithstanding FinTech and everything else that's going through but these days it's the opposite you know there are companies starting up which don't need capital at all and the fact that you're deploying the capital on a five-year cycle or ten-year cycle is not an advantage to you because they're not they are operating on a six-month cycle they're moving faster they're innovating faster and I think when you look at these the you know the impact of how that dynamic plays out in this sort of environment that's where it becomes fascinating is to understand what these actually small companies are doing in this space for other games and I think that's actually some of the most interesting aspects to this question absolutely so we don't know well we don't we don't and the other thing which is what grace started out with was the Airbnb being the largest hotel accommodation accommodation company without having any real estate at all so both of you are saying exactly right sorry then going to the next question perfect timing to the next question which is how is artificial intelligence decision-making and human bias basically looking at the two things and what are the ethical implications I think this is little oops the question that was asked by yourself is that right so that's that's the question to the panel do you want me to repeat it again are we good at their co-worker implications of AI decision-making I guess that's the question well I mean I think I think this is an area where we all have to work together but we trade and the question was asked privacy for convenience every single day every every time you know we order an uber car we're telling him exactly where we are right I mean so we do it but we have to look at some of the ethical considerations I was I was in New York City about was when I first started with IBM and we were with some of the best of the best in medicine some of the car manufacturers and we said we have to work together from an overall industry standpoint relative to the ethical considerations which is why we reached out to the Microsoft's and the googles of the world because you know at the end of the day it's what are we doing with this technology how can it be used and how can we use it for good and and I do believe that this has an opportunity to transform how we're doing things from an overall risk management perspective in terms of mitigating risk and making processes that were manual and resource intensive more efficient and effective but I also think that there's some economic benefits in terms of being able to make more money because the more we know about our customers the better client experience we give the more apps they are to continue to work with us so I think the good outweighs any of the negatives also believe that it's about augmenting going back more Martin and I are totally in alignment and it also has to be about transparency one of the earlier presentations was about a black box you know if a black box for credit scoring I don't believe in black boxes I don't believe in black boxes if these systems are one big black box and there's no transparency associated with it and no audit trail and no ability to know how certain decisions were made that is not where we need to be from an industry standpoint it has to be about transparency it has to be about audit trail because in my space and financial services is related to risk management how did you get to that decision regulators can justify it you have to be able to absolutely no just a question - Martin on the back of that why is there so much media hype about the fact that you know the ethics are not being accounted for in the big companies a still that much media hub that we don't at Hanford let's take the above this yes there is firstly it's not so I'm saying there are we were actually and you say that we're not an abyss where we can actually be friends and especially on these type of questions we do need to work together account explain the media hype I'm not going to get into how somebody had to say the cab driver brought it up to me whenever I was in the cab on the way to the hotel not going to go into all of that about that it's one of those things we do see it coming and we don't want to have an operand haim effect where we look back and see we would have never thought that this came out of it because this is not something that's going to be one of you bet it's going to be exactly like you said redefining industries redefining society redefining how we actually consume information which is the basis of everything we do not just in risk management and they're just two points from your very important here the one is we've seen I didn't know the lecture left at the beginning here this lovely diagram about where could we go when we have different credit scoring and if you overlay this was different societal patterns I'm not going to take an opinion of this but this is exactly the debate we need to have what however if we open this up if we take regulators and regulators we already see it in financial modeling frtb both mentioned at some beginning they go into models how should we model things so they're not going to stop looking into just financial models they're going to start and actually auditing and looking into different AI motors and we already have these discussions while they want to see like open the black box if this becomes difficult when you look into newer networks but these discussions are happening but what's the next step if we if you say well we don't like this social outcome we always have this mantra in in our machine learning that today to speak which is absolutely correct but what if we don't like what the data's say so we're going to go and tweak the data there was a lovely quote if you torture data enough it's going to confess to anything we can do that or we can simply overwater we had a lot of events in history starting from financial crisis very recently going all the way back in a frank presentation there were a lot of examples there were actually humans overruled and we might think this is for the social good and this is a valid debate I hope I can't have an answer but these are the debates we need to have if the overall if we step in this is actually going to help is it not going to just introduce another bias and this actually leads to the second poll that is very important for me here we've got big companies sitting here it's very easy to say especially in finest well let the big companies do it or we'll have to regulate to do it but this is one of those debates irrespective of banking violence anything AI machine learning where the responsibility goes out to everyone this is not just about what we deal with it for better or worse we're living in capitalism and we are driven by what the customers want so suddenly people start saying well we want this AI am that I do everything you for us just make us automatons that do whatever my smartphone tells me well this is not for us to decide this is where everybody in society needs to say how far do we want this to go how much data do we want to give we can't wait for regulators Google Microsoft IBM to solve this so these are the things where it's good to have these events well absolutely and you're thinking about it that's the big thing well you know to have the three of you four agree to talk about it and thinking about it yes clearly not the five biggest brains in it companies doesn't get as much as putting everybody in one room and getting actually traffic you answer my question at least the hype is in my opinion our hype man Frank do you want to add to that in terms of ethics coming from health care or well I you know definitely in health care it's a big thing obviously I mean if you take a risk and you lose money that's a big thing if you take a risk and you lose the life that's a bigger thing I mean let's be very clear about that but I would also like to say that you know as a great philosopher once said let's not be afraid of the world we made there was Billy Idol in one of us it's hoping for some right but it is you know listen I mean we are the Imperial College here yep science is gonna progress science cannot be stopped this stuff is coming this stuff so we need to take we need to take this responsibility and deal with it and and take the right steps and do it you got also realize that your world is already completely governed by this stuff I mean there is so much so out there I mean chemical plants were run by this kind of stuff forty years or now maybe 25 years ago you know the intelligence in there that says don't do that because it's going to go boom the engineers program that in they don't let an operator make that decision so so we need to go with that but it's got to be a combination of the human intelligence and the artificial intelligence you've heard that over and over again that's my point thank you thank you Matt so you know I think this when you look at a machine I mean we talk about AI first and and you asked about this or how people think about it how it relates to society and I'd say of all the technologies that we've seen in recent years ai probably has the most mythical aspect to it what do I mean by that I mean that within our culture and our psyche the idea of the robot the Terminator Oh or lady goes another one and I mean you can trace these kind of missed a whole tumblr about two and a half thousand years to Greek times you know it's really this founding idea of what happens when we build something more intelligent that we are so when we're talking about artificial times of machine learning you've got to recognize there's a mythical dimension to this yes on which we pin hopes or fears now if we'd reduce that lap back a little bit and look at what's actually going on in the area of machine learning and then within that there's two real areas as those supervised model developments where what the AI trains on is data that was given and decisions that were made it's important and the ethics of that AI and its moral compass is really based on the training set that it's provided with absolutely now it sort of gets a little bit more interesting when we need to when we look at the idea of accountability and so you know how many people here have been on a jet plane transatlantic around Europe recently and how much work you think the average pilot does these day hurdle is and House of the Year no it's almost all autopilot and has been for a long long time but we always have a pilot in fact know we have two and the reason for that is actually the dilemma of accountability how do we explain our actions and stand by them and I think this is a fascinating area the deep mind guys actually published in papers a couple of years a couple of months ago about being able to look inside your networks and have them explain themselves and what's driving that it's a lot of work that actually doing in the health care area but they're doing in financial regulatory areas where you have a model that needs to explain its decision and give the reasons why and I think that's going to be a massive area of research over the coming years because it feeds into these critical aspects of accountability and mech turning what could be seen as a black box into a white box and you know if you want to track some cutting-edge research these are the sort of areas which are fascinating right now well like patterns potentially to identify cancers better and ecologist which you were referring to as well better risk management more efficiency more effectiveness but but going back to the credit risk example also you know banks have fair and responsible banking departments fair and responsible lending departments so if people are going to be potentially pushed out they're looking at those things from a statistical perspective and saying this would not be appropriate I mean the you know in corporate compliance organizations and in first line of defense that exists so there are controls in place to mitigate some of those concerns associated with what's what's being brought up thank you hey so the final question for the panel and that a I rewrite Black Swan Theory no well Frank do you want expand on don't take your time I mean black swans are what they are right I mean we you know they are called black swans they described by Nicholas Taleb in his book and and and quite extensively and why you should think about them and what we've done honestly is we've called everything that was a surprise of Black Swan yes now just because you're blind and you don't see something doesn't make it a Black Swan so a lot of those will go away as we have better forecasting I I would point towards the book anti fragile which actually deals with dealing with the black swans and I would say that AI is beautifully aligned with with depth okay Raul actually just second and I think we have a normal graceful once the black swans to be perfectly honest and we had a couple of these things companies coming to us and saying we missed the past five years of trance and there was food industry there wasn't anything in finance and today with the data we have a new show the graphic and you showed on the appliances multiplying into multiply those two together and then you you have a completely overwhelming store of data we took all of that data and we looked at it we got the five trends right on the back we gave one to the front they sent the team out there to investigate was a fifty billion dollar market missed it in the u.s. at the moment to your point it wasn't the Black Swan it's just that today while we do this augmentation right we have so many eyes so many is more than we used to have before but things that are just completely unpredictable we're to your point of the I think it was Abilene the the a plane accident well apart it did have the history of mental problems and mental problems not for everything that exists and we will never get those away nothing today as you said we're calling so many grades Wars plague and hide behind that and well we could just not have known I could not have known that there was a rogue trader doing a lot of hundred and thousands of trades that every is just impossible to we're gonna have this excuse anymore with the technology for the data we have so we will see a lot less or the theory I believe still stands right but can you sort of I know you won't want to say this but can we move towards saying that we need to get less colorblind or become more colorblind which way you want to go I don't know but basically does it really is it really relevant anymore really and the grand scheme of things electrons yes what do you think about Fukushima Elon Musk and fracking did for the Amazon so does those re real black swans how are you going to predict those with artificial intelligence but you can get resilience the whole thing back to what Matt said which is it learns you're learning with the the deep dive that you're talking about that it's actually happening it gives a different sense I still think you you won't get away from from them them being that I really don't I really don't think you're gonna get this actually to your point we might rather see a couple of new black swans we had our own learning experience but we put a iPods up and suddenly they behaved in ways that were some people know there were no they expected nor well they're decent it we took them off again you don't know you have to just learn to fail fast through these things as you do in a startup environment but in any case I don't think we got to get rid of black swans I don't know when you asked about whether it's relevant has it ever been relevant as accepting that are things we can't control that is just accepting it and be done with it right but I do believe we need to take seriously that we can do much better than we do today and we need to be prepared for things to come out of what we're doing today that might actually come back to bite us in the butt deterministic from what I learned from math today in terms of our modeling it'll be less probabilistic and more deterministic who said that somebody said that so it is a bit more definitive I was I was at a large technology company who processed about 60% of the mortgages in the United States and we watched people consumers using their home equity loans to buy dinner we saw it we all saw it we knew it coming we knew it was coming when we do anything when you look at the home prices continuing to go up and the fact that income was not at a sustainable rate to match up with what was happening and the price is continuing to appreciate and people using the ATM or the home as a piggy bank relative to refinance refinance refinance take the cash out take the cash out take the cash out we knew what was gonna happen and a few people bet against the market and they made it from the Menace amount of money did very well but most people did not see it coming because of the bias because prices are gonna continue to go up and they can continue to go refi and it'll be okay and then it wasn't I think I think grace makes a key point there actually absolutely yeah she remembers the the the the real story behind the Black Swan and why it's called this you know the term Black Swan was invented in the 16th century or something like that and it was invented and used to talk about something which was everyone agreed was impossible because it was no such thing as a Black Swan yeah until they sell to the new world and found something and you know the motif really is that we make assumptions about stuff that we think are absolutely true but they're not and that's what catches is out so when you think about when we're training models or we're training systems we're implicitly making assumptions that we're not accounting for and so it is a very interesting way of looking at this is a paper I read probably eighteen months ago and it was talking about different kinds of uncertainty and it laid them out in this ways you know the stochastic uncertainty you can apply on this is you can fire guerrillas to those and statistics to those and you'll get bell curves and everything else and you can treat the toughest Augusta Conservancy probability they said that's a very different order of uncertainty to the next category which is systems so since you get very complex and multi-dimensional systems you can't treat them statistically there's too many variables and actually you don't get right answers because the inputs you make to a system affect the system and affect its behavior and so you see that in natural systems you see that in in large complex systems and actually machine learning can start addressing those problems as a technique because it has this adaptive quality can take multiple variables but the fourth category is when you've got people on the other end and this is where the bias comes in because you know it's great is it's very hard to convince someone of something when their salary depends on not understanding it yeah and if what you've got on the other side of that risk it's the one who's actively trying to understand what you're doing and frustrated and you make the mistake of thinking it's a different category uncertainty you're done they've got you and so when you think about everything from military tactics and military situations you've got to be really careful about what sort of uncertainty is actually at their end of it otherwise you dead so we've just got to be careful I think of not trying to believe this there's going to be a silver bullet for everything it won't be people are intelligent they're creative they understand what you're doing and they're gonna play poker with you thank you so much thank you very much power that's great

Channel: Institute of Risk Management (IRM)

Views: 5,606

Rating: 4.8823528 out of 5

Keywords: ai, artificial intelligence, imperial college, fintech, trestle, ibm, google, microsoft, 4th-ir, risk, risk management, technology

Id: _FFzYkcI8UI

Length: 79min 11sec (4751 seconds)

Published: Tue Dec 19 2017