[Fireside Chat H2Oai + WellsFargo] Model Security and Validation in the Financial Industry

hello and welcome everyone thank you for joining our fireside chat with sri ambadi ceo and founder at eight show.ai and agus zugindo vp and head of corporate model risk at wells fargo we'll discuss model safety and validation and how wells fargo has been using h2o wave before i hand it over to sri i'd like to share that we'll have a few minutes for q a towards the end of the conversation so please feel free to send us your questions throughout the session via the q a tag on zoom or via the comments section on linkedin twitch or twitter so welcome shree and a goose thank you bruno thank you uh good morning good afternoon good night good evening for people on the uh uh online thank you thanks uh a goose for um taking the time to spend with our community and h2o super excited um for some fireworks so um we can get started uh with lots of good gamut of really interesting topics on the agenda between interpretability through how covid has disrupted modelers lives and how model validation can be a powerful way to bring uh sanity to the models that have gone completely wrong um without further ado i wanna have your comment about why model validation i think i heard a quote from you model validation is not software qa yeah yeah well the uh i think this is probably uh model validation and independent model of validation came from the financial world writer practice in the financial institution uh as a very regulated entity in the us the institution need to have an independent group that's doing model validation completely independent from model developer and they have to report management path management reporting chains are different to make sure that they are independent now when we start talking about model we know and we believe all models are wrong right three all more are wrong they are useful but all of them are wrong and and and but they are useful okay uh knowing that using model that means we know we use we are taking risk because the model will be wrong and can be wrong and when models are wrong or doing something wrong or unintended has unintended consequence it creates damage it creates harm to either the customer or to the institution itself so it's very very important when we build model we develop model we we deploy model we know how the model will be wrong and what to do about it has the risk mitigation so to me model validation is the equivalent of reliability engineer or safety engineer in product design so i used to design car engine for ford motor company right before my life in in banking so we have a design engineer who design product and we have reliability engineer and safety engineer who test the product that's the situation in the environment where where the product will fail and if there any safety concern so when we look at model validation it's not simply a qa exercise similar to a software qa it's not that at all right because unlike software is deterministic model is probabilistic right you have probability of of wrong when when software is deterministic so understanding that and how the model will be wrong and what are the source that create the model will be wrong and tested independently so the focus of model validation is to test not on the performance but to test to fail the model right so model developer rightly so need to focus on model performance optimizing model performance model validation will focus i'm going to make the model wrong what situation what's environment what scenario what's usage condition uh model will be wrong and where that can be wrong it can't be from data input can be all kind of things so coffee 19 is perfect example of this right the environment completely different completely changed and the input that used to be they used to be the input in the normal way becoming super super abnormal right the uh who would have thought that credit quality in the usa credit quality during coffee 19 time everybody created quality credit scores going up it's because nobody has allowed to be flagged as default so nobody default nobody delinquent because of forbearance and all of those things so so that's the thinking in terms of how the model can be wrong in what situation and what source of the model and how to mitigate them and that's that's really the focus of modern validation i think you said covet 19 is a perfect example of the lack of imagination to test the model's boundary conditions um how could we have prepared better well i think this is uh bring to the uh uh coffee 19 is probably the extreme un unusual environment right but in general even during the normal usage we need to think about the issue of model robustness right uh the model world is not perfect so when we built model in machine learning world we split data training data testing data and we pick model that's the best based on testing data well that's a perfect environment because your data doesn't change your data is static right and in real world the uh the input is dynamically changing and coffee 19 is perfect example it's the completely changed in even the model c some the model has to operate in environment that they never seen before in the training data so in term of lack of imagination is we should really so that we don't have a problem like what we encounter many situations that we encounter during coffee 19 when they hate the first time is really thinking about in what situation the model is going to operate and test it that's the robustness under the situation so we know probably model will fail but at least we know it up on how the model will fail and how we're going to how we're going to deal with when the model will fail so i think i think this is very important and now i'm bringing the issue of safety here again and uh when when uh when we died designed car engine long time ago right uh we have uh we we we we can shut down the engine to open it in olympic mode the throttle body will not cannot be openly wide open so that uh it's going to uh you can drive it with limping mode they will go slow you know in a safety mode so it's a so i think it's very important to understand that so because when your model failed you know it's going to get exposed to uh to a certain environment when the model never gets exposed the model should be either either smart enough or have a a safety situation where it's operated in olympic mode so that it will not create harm i think one of the things that the team has been working on is actually the h2o ai app store as you know and one of the apps that the team has worked on is adversarial testing ai to start testing um and building adversarial tests against ai so for example here we have a ability to connect to automatic machine learning like driverless ai and use it to start testing against models and start creating a very strong adversarial look at the boundaries create some real conditions against the target variable depth income ratio pick some interesting models columns to attack and then generate um what kind of rate at which you want to change the data in the ranges that are in the domain serial testing is going to become a more and more uh necessary uh function as opposed to a um good to have yeah yeah so i i would put it in the uh in the broader context right because you have adversarial testing or people sometimes call it counterfactual or in general it's really a uh a generalization of sensitivity analysis right so you perturb your input because hey this is the environment that i'm going to operate and then from the environment possible environment you start thinking about how are we going to perturb the input to reflect that kind of environment and in machine learning this is very very important and becoming very important because unlike linear model well the model is very very clear we know very easily the weakness and how robust the model will be the robust evaluation is easy in machine learning it's a lot more complicated because the model is non-linear or even if if linear it's it's locally linear so in overall is is highly non-linear so it's becoming a lot more complicated and having exhaustive testing also very difficult so it's uh i think it's a very very uh very important area so that we know what is the limit of the model robustness of the model in fact sometime when we choose model uh that we want to deploy we may not choose the model that has the best performance in the situation of standard or ideal situation performance we may choose model that in the in the ideal situation from training testing is slightly worse but when you test it is a lot more robust yeah i think i think the idea is that making sure that the model is not deployed without advanced validation schemes around the scenarios and i think you mentioned um your early days in car safety how did that help you now your i think of you as the lord of all models over at wells fargo as the ep of corporate risk modeling how did that journey in looking at human safety in cars can help you in looking at safety of models and yeah uh i think the uh some of the practice that we we we it's a long time practice in the uh in in engineering right we we talk about in engineering we talk about tail testing we talk about tail testing mean we're going to test the product at the tail condition extreme situation how's the product is going to fail we need to understand that that's gives a lot of thinking in our our approach right how to do tail testing how to identify a tail situation where the where where model will fail so that's very important and then the concept that is very very powerful in engineering in the 90s in terms of robust design you want to design product that is robust or operate in the in the in the in the changing environment for example uh we test product in the uh in high elevation in uh in colorado or in the heat of death valley in the humidity of uh of miami right so all kind of things product operate that way so model is similar way model operate under different situation facing different uh subject and all of those things so in some situation product model cannot be retrained all the time for example credit model credit model has to be robust we originate credit and has to be good through the cycle in a good time and in a bad time so model cannot just react very very rapidly by by changing by very training yes marketing model probably you want to operate in that mode you know you retrain your model all the time and do all those things but in many many situations in financial world you cannot do that you have to have model that is robust so the thinking from engineering on the robust design on reliability tail tail testing that's uh have a humongous influence to uh to model validation excellent from model validation to robustness to then we want to look at how to understand these models when they are wrong they've gone wrong obviously debugging those models needs a lot of interesting techniques one such impressive uh method that came out of wells fargo through your uh to your uh team and your work has been uh is uh using locally linear methods if you love to kind of you're not gonna double click on that and hear more about um yeah yeah i i would i would before we go to that i would i would step back in terms of uh the the spectrum of where we apply machine learning from area that is giving alert to our investigator if it's uh for fraud or financial crime all the way to model that decide the livelihood of people approving or not approving loan right approving not approving loan approving for doing banking or not that decide the livelihood of people that's very very critical so with with all those uh spectrum we need to understand in terms of transparency interpretability of the model so this is very very important for us for many things that we do are very critical making decision of livelihood of people so model that is self-explanatory is very important model that is intrinsically interpretable now the question is can we have model that intrinsically interpretable but also high performance then yes the answer is yes we can we we can have our intrinsically interpretable self explanatory and high performance now because model particle machine learning is a complex and how the model can be wrong is very complex transparency truly understand what the model does and how the decision is made by model is very very important now uh sometimes people when talk about well gradient boosting machine is uh boosted three well it seems like very innocent it's a three it's very simple but when you have a thousand three it's not simple anymore right it's very very complex and it can be very opaque so you have to apply a post hoc interpretability like sharp lime and all of those things you do possible interpretability which is all right for area that is less in terms of high stake environment not deciding the livelihood of people when we start talking about deciding the livelihood of people then we really need very very interpretable model this is part of things that we work on how to create self-explanatory and interpretable model and deep learning ironically deep learning is the most interpretable model if we choose the right activation function in this case relu activation function because deep learning with relu activation function is basically local linear model it's a linear model just you have a thousand or it can be a million of them depending on how uh how complex the network is so that's what really motivates us to looking at how can we make a deep learning as a self-explanatory and highly interpretable thus we build a tool called alicia right and uh we put it in uh in reef with uh uh do uh to for people to have a better access so maybe you want to demonstrate this uh uh with alasia which is basically after you train a deep learning network after you train a deep network you can get all the local linear model uh from deep network it's an exact local linear model it's from the network it's not a post hoc or it's not an auxiliary tool but it's really what the network does unlike lime or sharp so see you are using the boston house price prediction here is an example so we put the neural network and then exactly understand what are all the local linear models so this is the parallel coordinate plot uh the uh of local linear model from deep network and you can get the feature importance which is exact feature importance so it's not sampling it's not approximation like lime or sharp because slime and sharp as good as they are they can be misleading they can be uh inexact and inconsistent where this one here is very consistent so you can have exactly the local linear model you can have you can plot the profile plot linear model along in its for each variable you can you can see it how linear non-linear the interaction when you see a cross line like cross line like that crossing that i mean a strong interaction for this variable because it's changing direction so it's a strong interaction different local linear model have different coefficient you can see how the density plot this is in wave right you can see the density plot how the variable get grouped by the network um we've seen a lot of uh density plots but this is uh incredibly beautifully done by the wave team and shivam and uh your team yoon zibin and you can see the global and local performance this is showing the uh uh because remember deep network especially a lot of local linear models so this is example of a few local linear models you can see what is the global performance in term of mse as well as local performance so this is a mse in this example global local performance the smaller the better locally the model is very very good that's why you see two bars if you can slide you see two bar i think the bar on the left is uh error for that local linear model so it's very very good globally is not good so that means that that's why you need to have uh this uh deep learning model right because linear model is not enough locally it's very good locally is enough that's what it's shown but globally when you apply all data it's not good so so this is indication that yes you need more sophisticated model because a lot of a lot of interaction among variables so you see it in the profile plot before when you see the cross line there's a lot of interaction that's you need the local linear model because uh what deep really does is basically local linear model and this gives give us a high confidence in term of the uh the model because we know exactly what the model does we can get all the local linear equation from the network and how the data is partitioned so so this is very very important uh for for diagnostic to understand how the model will be wrong what is the weakness of the model in what region the model is weakest right in this example here is the the tall bar on the blue bar right so that's uh meaning that well you you can look at the uh locally as well where where the model is the weakest so so that's that's all the the detailed diagnostic that you do so the model becoming really fully fully transparent self explainable because we don't apply additional tool here we're exactly just retrieving all the local linear model from from the uh unlike lime when you perturb and get local in your model this is not this is exactly the representation of deep learning it's incredible i think you've taken the adage of um think globally and act locally and apply that in in the fashion of fit globally but explained locally really quite uh quite a groundbreaking uh piece yeah and then you say as well when you keep learning uh probably before you do simplification three this is interesting to see in the deep learning yeah if you scroll down on the count on the on that table on the top right table the top ones which is because you drive it the top one here yeah if you scroll down i don't know if you can scroll down you see a lot of scroll down a lot of local in your model only have one sample or two sample if you have a regression with one sample or two sample can you trust it can you trust that local linear model i would not right and by the way when you train tip network if you don't regularize it and if you do early stopping you will get a lot of those a lot of local in your model with very very few samples in fact extreme only one sample and by the way also if you're cleaning network a lot of local linear model never seen any sample right so that's some of the danger when you use this so so one of this is we introduce the simplification right well let's simplify it can we get the same performance right but in in much much controlled way right so suddenly in this situation from uh the original 111 local linear model right into six local linear model with six local linear model now if you scroll down that table you see now the smallest h37 sample in this example so and this is the uh the the sixth local local linear model that you need you don't need you don't need more than that okay so six local linear model is good it's good enough and you know uh you have six local linear model that is you know exactly what they are it's very controllable this is the things that's very important in critical environment we know exactly when the model will fail right rather than not not knowing when the model will fail so in this six local linear model have high performance much much simplified model and we can control the failure and this is part of the learning as well from kofi 19. when things change we know how to deal with it because we know we have the peace that we already tame if you don't control it you have peace that you cannot control here we control it we control how the model will behave how the model will fail without sacrificing the performance and that's uh enabled by the uh the uh the alaska and the alaska wave that you can go inside your network and you can simplify it you can control the beast deep learning is very very powerful but if you don't control it it's a real beast some incredible incredible insights right there um let me scoot over for a quick question to shivam and team uh i know most of your team worked on this goes over the last um not that long actually maybe a week or so explain how how how does what is it what does it take to build a wave application oh yes uh so so i just to share like i am from data science side uh i don't have knowledge of javascript html or css so if i have if i had to learn all these technologies it might have taken me maybe three weeks or four weeks equivalent to one month maybe to build this type of an app where users can provide their interfaces there are visual components and so on but we have h2o wave which is our sdk with minimal code low code python based framework where we can use that framework to develop these applications very quickly and for this app a lithia app we were able to build this whole ai app within like four or five days the last was just polishing and just the proof reading but all the core work we were we managed to develop in like four days effort i think that's very powerful it's very impressive as if you know it just enable the data scientists right to uh to deploy application very very rapidly so they can focus on the data science right and and this is a a huge uh uh tools that you guys provide one app that actually even built is a favorite of mine where it's able to take a notebook on a kaggle notebook and automatically convert to a wave application so it's almost an app to build apps so um we allow for people to use this and start building more applications right sort of so um this is a this is just the beginning of uh the um the days of seeing a lot more um in terms of applications across the board so yeah so i mean i will um leave the team wave is open source and so you allow for world to start building applications and start generating and publishing them as well yeah as a data scientist jupiter notebooks are our favorite tool but one of the limitation is that we don't have interactive interfaces for those same notebooks so what we can do is we can convert those notebooks to interactive ai applications with more than static charts more than static outputs and then an app will represent all the work that we have added in the jupyter notebook that's where wave adds the value for all the data science work that data scientists have done let me uh double click on this particular chart agus i know boston housing data is relatively widely used one of the things it's showing is the human side of it right yeah do you want to comment on how yeah yeah this is uh probably uh a very popular data sci data set that is included in circuit learn but it's also very very controversial uh showing the ugly side of data or or data or data science right because in this example you can put it on the screen three on that one uh put it back on the screen on this example this is a boston house price right the most important variable here is i think is the ls stat and crm in there if i'm not mistaken the ls stat is the low income housing proportion of low uh low income housing in there you know so so that's a very very important variable of course that's the uh and then crime is the uh uh the crime side the crime rate right so it seems we have to in in the real world when we do a building model we have to be super super careful because some of the data and variable here they can be very discriminatory they are uh if if you're not very careful we're talking about the uh uh discriminatory aspect and the uh the uh the the ugly side okay the uh off of of of data and the way uh the society you know uh have been uh have been operating right if you're not careful understanding the source of data and when we build model that's where the dangers are because that's where we're dealing with fairness issue we're dealing with ethical issue in all of those things so the prerequisite of data science or any model building of course is really understanding the data what is the data tell us do we have any problem with that and do we want to even work on it and build model based on data that we know are very very biased so this is a big uh big topic big issue today as we uh in the past uh we our operating model is we have hypotheses we have things we look at the data to to to to get better understanding today in the in the world of machine learning the operating model is flip it's data driven we start from the data and let the data tell us instead of i have something i am going to look at the data to support my evidence now we let the data to tell us and we let model to make decision based on a model that is trained and built with on the data so it's an extremely extremely important subject to to uh really start with understanding the data what can you use what we cannot use what we should not use so that's that's a that's a very very uh important aspect and it if we don't control it that's creating what i call it before including in the model failure unintended outcome right unintended consequence model can create harm if we make a public policy we may do something based on this type of data that is a very very dangerous thing how does one fight um or balance for in some sense the inherent bias in historical decisions right historical data um are there ways one can mitigate it how do we create more robustness to to balance for it well i think it's a very very it depends on the situation and it can be very very challenging right because uh this is the uh it's it's also part of the uh societal norm right what acceptable what not acceptable what acceptable in singapore and in the us in asia and europe uh it's it's very very different so i think that's a a big challenge of course and i think a lot more now with with the regulation as well as with a lot of attention on this which is the right things to do to really uh do this so i think it was first of all as you uh you you have process in the company and also some of the ethical standard what you're going to do and what you're not going to do right so so i think need to start from there do i want to build models certain models so we can say no we're not going to to do that part because we know the model is uh is uh we have trouble uh it's societal parts and that so we're not going to you say to make decision uh it's uh uh it's uh it's sometimes it's a bit controversial in terms of do you want to to apply algorithms to to probably uh to to maybe sample it differently some people argue i can sample it differently right to to make it more fair or uh people can say uh i am going to train it differently so i'm going to have a certain criteria so you can suppress the importance of certain variable and other variables to be more more important so so it's it's it's less straightforward here uh but the the process and the uh the the due diligence that we need to do is very very important so having process to understand the data and what's the implication that's a starting point for us in wells fargo we have a very uh very pros clear process that our legal and compliance partner need to review because they are the expert on this they need to review what variable we can or we cannot use and then from that model developer can do the the job to build model and then they're going to test it with variable importance and testing in the mapias fairness testing that we do model validation will independently test it as well get the outcome and then from the outcome discuss again with our legal and compliance partner so this is what uh what's our finding sometimes it's very very difficult decision sometimes it's very straightforward because they have rule and raw rule and regulation in uh in in credit we have the uh ecoa right we have rank b who is very black and white in terms of what you can and you cannot do so that is more straightforward but other area is uh can be more difficult for example i give example in uh everybody do a lot of voice to text translation right voice to text and when we start talking voice to text a model will not be perfect some people some ethnic group coming from certain geography will be more difficult for tomorrow so it says a lot of miss translation from from voice to text is it acceptable is it not so we have to be very very careful to really understand what's the outcome and the implication of that we people can say well the model is useful it could help to to uh to to to to serve customers better but is it because because it serve customer better in certain area it may be it may not be uh better or maybe not as good as in other uh for other type of customer so it's a lot of a lot of uh discussion that that we have to to to deal with when we start uh applying machine learning for for this uh more complicated situation i think there was one other thing that the team was actually beginning to work on and one of my favorite topics actually is how nlp needs needs explanation as well and i think um the team is um racing to to put something together but i think uh we'd love to hear your thoughts on sure sure well uh in the uh this is a special uh application in tax classification we do a lot of tax classification in the company right because we we do uh uh complain customer complaint how can we process customer complaint better and route it to the right people and etc so that complaint can be resolved very very quickly so we do a lot of tax classification in our surveillance monitoring we do tax classification as well so it's probably one of the most applied in banking is really in the nlp world is the tax classification now when we do that classification and if we did and interpretability is very important too uh for many many reasons including unintended consequences i'll talk about because language is uh it's uh somewhat biased so we have to be careful to understand it as well so a part of thing that we do that we like to do is we use uh cnn convolutional neural network so the convolutional layer we use it to do feature extraction so we have n-gram you have unigram bi-gram trigram so so the convolutional layer can be constructed to there to represent the uh the the the n-gram so the key is in in any of this and when we talk start talking about nlp when we talk about interpretable model we have two things one is interpretable features when the the output from max pooling of cnn need to be interpretable right that's interpretable features once we have interpretable features the output from from convolution layer go to fully connected layer we need to have interpretable model now interpretable model is the thing that was you showed before using alesia you can make deep learning very interpretable that's interpretable model so we have two parts here interpretable features and then interpretable model so that's what we do to have to uh to deal with both uh to have in the preferential interval in the problem model so interpretable features is very important in nlp in particular because we're dealing with embedded word embedding right we from uh one hot representation of word we create which we translate it into word embedding when we have word embedding the interpretability disappear if you have one hot encoding you know which variable or which word are important but once you put it in word embedding it's not it's the the interpretability is gone it's lost because it's distributed representation for each word that's the uh the trick that we do on the on the cnn how to go through the n-gram structure on the convolutional network so that from word embedding that's not interpretable becoming uh interpretable feature so so read our paper on that and the new paper will come out to the paper that is published out there it says interpretable feature and then do uh uh deep network using shop but we also have the one that is really uh uh interpretable feature and then use the uh interpretable using using alesia this is incredible um there's a good set of q and a in the q and a section i'm going to point to latia's publication on the about page as well and a lot of interesting tool chain also built right sort of self explainable ml um in on github h2o wave ways to contribute to this um please reach out to shibam or hakus or me and we would love to open this up for community to kind of contribute and improve explainability is not not an option it is a must-have what i understand from abu's comments today people's livelihoods are being determined by automatic machine learning and machine learning and as data scientists we owe it to ourselves to make sure that we we bring transparency to these patients i will start with some of these questions um agusta many of them are are actually around explainability um kind of um what is in your view using semi-supervised approaches for credit modeling is that too risky and uh my question is why do we want to do some ice sacrifice when you can do supervise right the data is there data is available right the only reason that you do a semi supervised is you feel like okay i i don't have enough i don't have enough data or maybe you if you want to do somewhat somewhat supervised if you say uh i can i can make the model more robust for example let me this is very popular things like people do like how to make the model more robust well i'm going to perturb the data and i'm going to train the model with the perturbed data right so so that's probably we can do something like that but i i if if you i i don't know i'm in the cam if you don't have data just don't do a complicated model do a simple model as possible i think this is the problem as well that people want to use a bigger and bigger neural networks well this is my personal opinion if i have very big model that can do wonderful thing i am not impressed i am impressed with i can have small model that can do wonderful things so i think that's uh i am very impressed this is why part of the alesia wave you see h3 it has simplification i can simplify it i don't need this deep big deep learning because it's uh it's on this small model are very important this is one thing that's very interesting as well explainability and using alicia is very important a while back remember a while back in the new rips has a competition sponsored by fico that professor rudin from duke 1 on the interpretable model i think if i'm not mistaken uh promiser rudin won with either uh logistic regression or three i don't exactly remember i think lost his regression when you run alesia on that fico data and then you hit simplify it will get simplified into a single model which is logistic regression the data is just linear so you don't need the complicated stuff so this is what what we would like to do as a data scientist right so we apply yes fine apply complicated machinery deep network or whatever and then let's look at the structure then when we hit that's why we provide the tool on the simplification in all acis you can simplify it when many ways to simplify it of course you can simplify to through regularization in deep network you can apply uh with decay like basically l to penalty or if you want to really simplify it you apply l1 penalty right uh which is slash su that's the uh uh rob tips irani and trevor has this stuff right you apply lasso l1 penalty in any deep learning package it says l1 penalty two you can apply that's that when you run alicia you can compare the one with l1 penalty and the one with without l1 penalty which is the best early stopping or or or drop off okay i'll drop out you can compare the number of local linear model across those things and then you hit simplify you can simplify it how much simplified do you do you a sacrifice performance or not we'll be surprised depending on the data some situation not always in some situations simpler model is better but not always okay so so depend on that but at least we have as part of model builder we we we need to be responsible to understand that to control at the very least controlling the beast controlling what how the model will fail and really understand the model one question that came up is around how do you think about third-party embedded models that are coming in say in a a more complex software say on a voice recognition system or an lp model and added into more complex software yeah we uh we test it as well so all the third party model in wells fargo has to be tested the same like internally developed model so you may not be able to see the uh the inside how it works but still you can do a lot of testing this is the things that the uh my bring back to my graduate school day right and uh you can do system identification right you do input output you can do design of experiment on the input and get the output you can do that you can do counterfactual testing the robustness testing that you you have right so you do counterfactual type of testing to test to test that so it's a lot more limited in term of what you can do because you don't do you don't understand what's what's inside it but it doesn't mean that you cannot test it and a lot of a lot of sophisticated testing that we we can apply that's what we did okay so and we also looking at how is it if it's compared so i think the key is third party do you know how the model will fail right so we test it we have to test it we cannot just we cannot just deploy because at the end of the day when they fail is our responsibility so test for failure and there are many ways to test for failure so you test it more like a black box right so of course you guys they like the black box but still you can do that thing no i think one of the things that um we've been looking at in addition to model validation is um methods like um back testing right sort of how do you start building some back testing automatically for the models or do you continuously look for drift detecting drift um here's a user most of the stuff is automatically built for models that are being run in the environment so people can essentially even if the models start um terribly they end up better and you can see that how the variable importance is changing yeah yeah drift detection much of this is inspired by the incredible work you're doing uh in model validation ugoos and some of the conversations you've had but our vision on making the environments more continuously learning in an app store way so people can build their own applications roll their own app stores even so there's a h2s core apps here but we expect our customers to build their own app stores or ai for good app stores and go into finance hopefully with the beginning of way we hope to inspire more applications inside wells fargo and take it not just to our customers but their customers and because i think apps are much easier way to understand and use ai and that's kind of our team here with h2o ai cloud we're super excited for the journey here and our customers have been the true true guiding force for us in the innovation um especially around explainability and robustness your work is um groundbreaking so we're super excited to partner with your team on this more questions that are here are around how do you um communicate when regulations so the battle between regulations and ethically right decisions when regulations allow it and when it is not the right thing to do how do we communicate that to all stakeholders i i have a slightly different view you know uh i live in regulated uh entity right so if you look at it a lot of check and balance will happen in banks because this is a systemically important institution right so if something happened with the systemically important institution like banks it has a big impact to the uh to the society and the uh economy of the country as as a whole so so so with that check and balance is very very important for example in our case yeah think about the regulation in the in the financial world we have the uh we call it a multiple line of defense the front front line are people who build model and deploy model and use model right and then you have second line which is risk management right who oversight to test it independent testing and all of this thing and then we have our audit team who testing the second line and the first line okay so that's the third line and then we have the fourth line actually our external auditor right we have external auditor who check what uh what we have done and then we have the fifth checker actually which is the regulator some of you that's not familiar with financial institution uh occ and fdn and and and frb both of them they are equipped with a lot of phds in this in in in quantitative world phd in maths that and all those things they coming and they check our model so it's a check and balance and and and depend on the uh how important and how critical and when the model is wrong what is the harm so i think it depends on how harmful the model can be that's when the rigor or the regulation or whatever so i i i communicate a lot with the regulator they are extremely extremely very very sensible when you when they when they can when when you when you are competent you know so i i never have any any any things that really we occasionally we have disagreement but they are make sense they they are in the interest of protecting the country they are in the interest of protecting the customer as a whole right so i think that's uh that's very very important and they have been very they they bring very very good view and if we are competent we know what to what to how to explain we really understand it we can explain and they they listen to it excellent thank you so much um for incredible words of wisdom and um i actually loved the concept of simple models right sort of um during covet one of the best things we could do was build simple models because the times were changing so fast data was changing fast it's important to keep track of what really is happening in the data the ability to see through the model because simple one was so influential and it's it's a it's a responsibility and sustainability too right three it's not burning the uh the energy right so i when i talk about some other area for example some of the bitcoin mining and probably the biggest bitcoin mining happen in uh in china and they are the biggest coal burner right so i think in the machine learning world we are the same way you know our model becoming bigger and bigger and all those things i i think it's not the bigger model is the better okay i i feel like the simplest model uh is in in in that world but also simpler model that's more tractable it's uh with high performance we don't want to sacrifice the performance right uh but the uh that's it's very important because we we we can control when we understand how the model will fail i think it's the the most irresponsible things and we deploy model without understanding how the model will fail and how the model will harm our customer that's the most irresponsible thing i think customers are creating great customer experiences is exactly why we are here and ai should be really in service of making that happen and um explaining it debugging it understanding it understanding the data all of that is aided by by simplicity um i know you're out in the wilderness track um climbing mountains and seeing simple things in the nature um [Laughter] tell us about um where you are not you're obviously not in front of your office which i recognize from [Music] [Laughter] [Music] well i uh i try to enjoy the nature here i live in north carolina we have a beautiful scenery mountain is two hours away from us so we have beautiful places mountain gorges river waterfall anything so i try to enjoy as much as possible when i'm not writing algorithm or thinking about math i'm trying to you know going in the middle of the middle of the booth without any foreign connection just in your nature right especially important nowadays during the coffee 19 when when we cannot travel a lot i always enjoy travel uh around the world now i cannot travel so uh i enjoy what's uh really what's in the backyard now nature nature brings that simplicity and like nature h2o we are striving for that simplicity and we love our customers because they drive us to being excellent i think that's that's that's what i like about you guys you know you're always very uh customer-centric you talk to us a lot you don't listen to what what what we need and uh that has been uh uh tremendous in my view it's the team um you're part of our journey and so so excited and there's some so so many want more ones in the audience and the community and the customer love actually brought us this far so we're super thrilled to we have the opportunity to serve um your your innovation to be honest and on lithia super excited to see being used everywhere uh globally so we uh we are keep pushing uh you keep pushing you know for us it's the the journey to have a high performance explainable uh self uh self explanatory intrinsically interpretable model you know because with all the noise what all the thing i feel like the explainability is very very important that's why we spend really really a lot of time on in that i i'm happy to be able to share with with with h2o and jointly working together with h2o to make some of the tools available for for people out there we call it makers going to make it's maker culture and we are so excited to to have a partner of equal caliber or greater caliber to make more beautiful things and explainable things intelligent things super excited thank you for joining us today thank you thank you everyone who has spent their time with us thank you luna and

Channel: H2O.ai

Views: 311

Rating: 5 out of 5

Keywords:

Id: -5_kFYEZmJc

Length: 58min 19sec (3499 seconds)

Published: Thu Apr 01 2021