Ian Goodfellow: Adversarial Machine Learning (ICLR 2019 invited talk)

hello everyone we're going to get started this afternoon welcome back my name is David I'll be your session chair for the afternoon so our first speaker a keynote for the afternoon I feel like in this community needs very little introduction in Goodfellow this is PhD completed in 2014 under yoshua bengio and erinkoval at the university of montreal he's perhaps best known as the lead author of the deep learning textbook by MIT press co-authored with his advisers Joshua and Aaron and he's done very influential work on adversarial machine learning both generative adversarial networks which are a well represented topic at this conference and also on adversarial training and methods for mitigating and detecting adversarial examples and he's going to give us a overview talk of adversarial machine learning please welcome the speaker [Applause] good afternoon thank you for coming I'm glad to see everybody here I'm going to try to give you a nice overview of how adversary scene lending relates to several different research topics today to start off by what do I mean by adversarial machine learning I want to contrast it to more traditional machine learning most machine learning algorithms not all of them can be described basically as optimization algorithms where we intend for the solution to the optimization problem to generalize so optimization means we have some kind of cost function shown by the z axis on this plot and some kind of parameters represented by the horizontal axis on this plot we want to find the minimum of that cost function or we at least want to reduce the value of the cost function to some very low value in adversarial machine learning we don't look to optimization as the mathematical language for our algorithms instead we look to game theory in game theory each player has their own cost and each player tries to reduce their cost but the other players choices also affect their cost the simplest version of this that we can visualize on a slide is a minimax zero-sum game where the two players costs always add up to one in that case you can represent it with a value function as shown on the 3d plot on the right to make this easy to visualize I've shown just one parameter for each player but you can see the value function is essentially this 3d surface where one player tries to minimize the value and the other player tries to maximize the value if you look at cross-sections through the cost function in terms of the parameters that each player can control player one is looking for a local minimum and player two is looking for a local maximum if they both find a point like that at the same time it's called a Nash equilibrium and we think of that as a solution to the game because neither player can improve their cost without controlling the other player so that's the main difference between traditional machine learning and adversarial machine learning until about 2013 nearly all of us working on machine learning and presenting papers here at AI clear and similar conferences basically all had the same goal in common we were all trying to get machine learning to work in most cases that meant we were working on supervised learning either directly or indirectly we worked in different application areas like speech or vision or text but we were all really just trying to get machine learning working at AI level now we more or less have supervised learning working given a large data set and we've seen a Cambrian explosion of different research topics where people have moved on to adding new capabilities like the ability to generate rather than just recognize inputs or the ability to learn from reward functions instead of supervision or adding extra properties like security or privacy now that we've had this Cambrian explosion of different research topics I think it's careful every day it's it's useful every now and then to take a step back and look at the big picture and see how some of the same techniques can actually be leveraged by a lot of different communities the first topic that I'll cover is generative modeling the goal of generative modeling is to take a training set of examples and learn the probability distribution that generated those examples the result can either be represented as a density function or it can be represented as a machine that can generate new samples from the same distribution or in some cases you can actually get both of those things today I'll focus on the ability to generate new samples from the distribution to illustrate how it works I've shown here a training set of photos of celebrities on the left this is the cell of a data set and on the right I've shown two new imaginary celebrities created by a generative model this is a progressive gun I was trained on celephaïs it inferred the distribution and it created these people who have never existed before but look as if they could realistically be celebrities who would be statistically the same as the real celebrities that we have today the way that Ganz work is actually via a two-player minimax game one player the generator creates images or other kinds of data the other player the discriminator recognizes these images as either real or fake when we train the two models we see that there's adversarial competition over how to classify the fake samples produced by the generator the generator tries to adapt the input to the discriminator to cause it to be misclassified and the discriminator tries to correctly classify each of the fake inputs as fake at the same time the discriminator face is a non-adversarial component where it learns to recognize the real examples as real using the language of game theory we can analyze this and find that there is a Nash equilibrium where the generator recovers the data distribution perfectly and the discriminator can do no better than random guessing about whether the input is real or fake in practice we don't usually reach that Nash equilibrium at least not with the algorithms we have today but we do get to the point where we produce realistic samples since the session chair and some other people and I introduced Ganz in 2014 they've improved rapidly you can see that back in 2014 we were just barely able to generate small grayscale faces from the Toronto face database and since then other people have found better and better approaches that are able to generate high resolution high-quality photos one thing that's interesting about this particular graphic is that the progression has been even more extreme than I can show you here the image on the left of the Toronto fate face database sample is actually scaled up to make it visible on the slide and the images on the right from 2017 and 2018 have both been scaled down dramatically so that they fit on the slide so if you could actually see the true scale of these the number of pixels has improved even more dramatically than this timeline implies generating faces was actually one of the easier challenges for generative models generating very many different categories turned out to be harder it wasn't until 2016 that Augustus Edina and his collaborators were able to make again that could produce recognizable samples from each of the different imagenet classes even then they were not very high quality samples but after we had the initial recipe we made rapid progress until big Gann was able to make high-quality high resolution samples from all of the imagenet categories presented this year at AI clear one thing that's especially exciting about Ganz is that they make it possible to learn with less supervision than many other machine learning algorithms are able to do here we have again that turns day scenes into night scenes this approach is called unsupervised image to image translation the way that we train this model is actually not by collecting pairs of day images and night images if you think about it it would be really to collect such a pair you'd have to go out on the road in the daytime see a bunch of cars on the road take a picture of them and then somehow rather get those cars to come back at night and read rive exactly the same trajectory so maybe in some 3d rendered synthetic environments you could do it but for real photos you can't the Gann framework makes it possible to Train the generator using feedback from the discriminator rather than from a specific target so you can think of it as a way of telling the generator that there are many acceptable outputs rather than one specific target provided by the supervision signal one of my favorite examples of unsupervised image to image translation with ganz is this cycle Gann video turning a horse into a zebra you can tell that it would be really hard to gather supervised horse and zebra images because you couldn't really go out in the field and twist the zebra into the horses position you know you jump in the photo yourself while you were out there awkwardly trying to get the zebra to stand how you want it to stand one thing that's interesting about this video is you can see some of the effects of bias and datasets on machine learning here it's very visually obvious that the background is changing along with the horse itself the machine learning model has no way of understanding that the horse and the zebra are the parts were interested in so the background is actually changing from looking like a horse environment to looking like a zebra environment another thing that's interesting about this video is that it's generated one frame at a time and the generator has no context about the the time before or the time that's going to come after so you can see that there are some artifacts in the video in particular watch when the horse turns its head at the end of his his trip across the field the zebra stripes remain stationary and the zebras head moves across them so watch right now right now the head moves across the stripes and the stripes stay still so can you actually improve that and make a video that's coherent it turns out that to make a coherent video you can actually generate the whole video sequence and feed it in its entirety to the discriminator and then the discriminator can tell if the whole thing is coherent manipulation and synthesis in addition to street scenes we show our network synthesizing videos for other domains for example our network can transform edge map videos to videos of human faces here we show some examples of synthesized people talking our network can generate different people speaking given the same input edgemax note that the results are temporally consistent from frame to frame another example we can synthesize videos of humans movie the impose information for example we can generate videos of people dancing this video of people dancing is created using a 3d model and then transforming the view of the 3d model into a view of the person dancing you could also have gotten that 3d model from a video of another person dancing so you could actually take a professional dancer and map their movements on - you know you yourself even if you're not a professional dancer that was accomplished by a team of researchers at Berkeley you [Music] [Applause] the video showed how we can take a professionals dance moves and then the user of the generative system can put themselves into the video using their movements it's also possible to go the other way around and take your own movements and your own design of the composition and have it translated into a photorealistic quality image it means that we can remove the creativity from actually producing realistic pixels - composing is seen and people who don't have the skill to be photorealistic artists in terms of the technical side can express themselves using began to accomplish the realism that they would otherwise have to train to produce this is called Gogan and then mountain in the back or cloud look nice how about we plant a tree on the hill like this note that the same label renders into two visually distinct objects the trunk and the leaves of the tree now listen so the Gogan image editor makes it possible to paint with pixels that say just which kind of class should appear at which point in the canvas and then a conditional gand that learns the distribution of her pixels given class labels per pixel turns it into a completed image for you so far everything I've shown you with Gans has been purely digital but one thing that has excited me recently is that Gans have started to make the jump into the physical world the first example of this that I know of is 3d printed dental crowns from Glidewell dentistry ordinarily a dental crown has to be customized for the patient's mouth by a human technician and it can take up to two weeks during which time the patient has to wear a temporary crown so it's actually necessary to perform two operations first to install the temporary crown and second to install the final one with the Gann generated crown it's possible to get a high-quality customized crown and have it 3d printed immediately so only one operation is needed I'd be excited to see a lot more applications of Gans to the physical world one application I'm really surprised hasn't happened yet but I've been predicting for a few years is Ganz for fashion we've seen Ganz generating fashion images but not actual clothing yet I know there's lots of people who'd like to put more pockets on their clothes and things like that I hope that genitive models make that possible soon so how do all of these work some important recent advances in how they work include the recent big gun model really invested a lot and scaling up on TPU and they found that rather than trying to stabilize again it's often better to train it with unstable hyper parameters and let it train until the last minute when it explodes and just save the last good check point before the explosion happened some other recent advances include the style gun from Nvidia where they incorporate style transfer techniques in a generation system itself we can see an example of how the style transfer works I show you three people's faces here on the Left we have a starting sample I want to emphasize even the starting sample is actually a sample from the ghin we're showing what happens when we changed the course style but the image did come out of the gun itself so let's say we want to style transfer on the course style of a different person such as a child the person in the middle is a child that was also generated by style gun we're going to take the course style of the child image and apply it to the image of the adult on the left the output is the image on the right which is a child version of the person on the left after transferring over the course style overall style gun makes it possible to represent a face in terms of course style where it captures things like age and gender medium style that captures some of the face geometry and fine style that captures things like the positions of individual hair or wrinkles in the face one thing that's been kind of disappointing about ganz is that we think of them as an unsupervised learning algorithm but actually most of the results have shown you required supervision to get really good samples from imagenet categories we actually have to train with class labels some recent work from brain Zurich shows that it's actually possible to reduce the number of labels needed to get a good image generation again in particular they were able to bring the number of labels required to match big-gun down to about 20% or sorry about 10% of the labeled data rather than requiring a label for every single image that's generative modeling the next major research area where adversarial machine learning has been useful is machine learning security so far machine learning has been very effective on naturally occurring iid data where all the data is drawn from the same distribution as we've used a training time and all the examples are generated independently from each other if you want to make machine learning secure and we want to make it so that an attacker can't intentionally cause the model to output a class of the attackers choice it's necessary to relax those iid assumptions we've seen many times over that it's very easy to do things like subtly modify an input and for example change this panda into a Gibbon according to the model so how can we actually resist this we also need to remember that it's possible for attackers to use very strange inputs such as graffiti on a stop sign or just present objects in a context different than the way that we see them in the training set for example this Apple is recognized by most current commercial object recognition algorithms until you put it inside a mesh bag and then even though it's not fully occluded it looks different enough than it did in the training set so relatively simple real-world modifications are enough for the iid assumptions to be broken and we need to think about all of these things we don't really have a full solution yet but so far a lot of the best solutions have been based on adversarial the idea behind adversarial training is again like with again to look for an equilibrium to a game you can think of one player as the machine learning model and it's trying to minimize its error rate as it classifies different inputs the other player is the attacker who tries to optimize to maximize the failure rate of the machine learning model by changing the input two inputs that will be misclassified another research area that I think has not been explored as much as I would have thought is model-based optimization and here adversarial machine learning will be very relevant the basic idea behind model-based optimization is that if you want to maximize some function that you don't directly have access to you can instead learn a model for that function and then maximize the models estimate of that function so let's say we wanted to make a really fast car we could make a model that looks at a blueprint for a car and then predicts how fast that car will be able to drive as we improve our blueprints we can move from something like the Model T to a really fast supercar unfortunately it's actually hard to do this today in a lot of different scenarios if you build a model and then search for inputs that are predicted to perform very well you'll actually get an adversarial example the model thinks will perform very well rather than a high quality example I think that some of the techniques in the gaen literature and from the adversarial example literature could help to unlock some of the potential of model-based optimization we have actually seen some applications of model based optimization already the best results that I know of are for designing new genes you can think of a gene as a sequence of DNA characters that describes a sequence of amino acids and when you convert from the DNA sequence to the amino acid sequence you get a protein the protein is just a molecule that actually does something in a Cell and to design new medicines what you usually want to do is design that protein so that it will bind very tightly to a specific receptor that's just a part of the cell that can accept these molecules and either gain or lose functionality based on having a protein bind to it so you can use model-based optimization to maximize the binding affinity of the protein by changing the DNA base pairs a few different labs have explored different techniques using adversarial machine learning to design proteins that bind very well in simulation I think this work will probably move on to actually making proteins that that bind well in reality soon and then we can start to see a versatile machine letting use to design useful medicines one research area where adversarial machine learning has always been relevant is actually reinforcement learning in fact you could say that the original machine learning was a form of adversarial machine learning one of the first machine learning projects that I'm aware of was Arthur Samuels checkers playing agent at the time that he made this in the 1950s most people believe that a computer could not do anything except what it had been explicitly programmed to do and his project demonstrated that that was not actually correct he wrote a program to play checkers against itself and learn from the checkers games that had played against itself because it had this repeated self play experience it eventually learned more about checkers that he himself knew so his own creation could beat him at checkers and that effectively proved by example that you really could get computers to do things that their programmers could not explicitly tell them to do this self approach is essentially a form of the minimax game that in recent years we've extended to other things like defenses against adverse early examples and extended to games it's a relatively simple form of the game because it's perfectly symmetric the same player is playing both sides and a lot of the recent advances in this field have actually been moving to non-symmetric games and thinking of how to do things other than just win the game for example Ganz it's not so important that we're winning or losing in the game it matters that we're producing high quality samples as a side-effect we've seen that self play continues to be a really important strategy for reinforcement learning though things like alphago and open areas defense against the ancient spot our boat based on self play and so are the sumo wrestlers from open AI that learned behaviors like ducking and faking out their opponents without needing to be told that those are useful behaviors they just learned them from self play it's also possible to make adversarial examples for learning Santi Quang showed that you can make adversarial perturbations of frames of Atari video games and you can even hypnotize the agent and make it perform worse in the future so you can show it a perturbation and then it doesn't make a mistake immediately but a few frames later it makes a mistake because the perturbation encoded something in it's recurrent memory a major challenge in reinforcement learning is to encode good reward functions that describe the behaviors that we really want it can be hard for a programmer to design something by hand that really encourages exactly the behaviors we want without encouraging side effects one way out of this situation is to learn the reward function and a recent project from deep mind showed that it's possible to use again like model to learn a useful reward function in this case what they want to do is draw specific characters on a canvas the characters are drawn from the Omniglot dataset which contains symbols or many different languages this little curly symbol here is one of these Omniglot characters we want to do in this case is have the robot paint that character on the canvas we can't just use a reward function like mean squared error between the photo and the canvas because the canvas is prevented as a is presented as a perfectly black-and-white bitmap and is viewed perfectly access on the canvas is viewed at an angle and because it's a photo it has lighting effects and things like that you're never going to get the photo to exactly match the bitmap input but you can learn something resembling again discriminator that tells you how close you are to accomplishing the goal and then follow the gradient of that discriminator instead of the original mean squared error loss and then that can result in the robot successfully painting the figure on the canvas another important research topic is extreme reliability a lot of our machine learning algorithms can get up to something like 99% accuracy but for a lot of applications that isn't good enough and in the long run we need to figure out how to make them even more reliable if we were going to use machine learning for something like air traffic control or a surgery robot you really want it to be more robust than the current models we have I think that adversarial machine learning can be a useful framework for eventually tackling these problems the reason is that adversarial machine learning is about solving for the worst case where you take a really messy real-world problem and instead of trying to model specifically how the real world is messy you just say what if all of the messy factors were as bad as they could possibly be because modeling the worst case is often theoretically simpler than modeling the real world case I've tried to illustrate that with a graph here with a blue curve showing worst case worst case performance and a green curve showing average case performance and a hypothetical problem the green curve is really messy and unpredictable but the blue curve is relatively smooth you could imagine that we could optimize the blue curve and then we're guaranteed that the green curve will be even better than whatever we got by optimizing the blue curve this kind of worst-case analysis and worst case design is a common engineering principle that's worked well in other disciplines one example is distributed systems at companies like Google that implement large file systems using multiple machines networked together a lot of that stuff is based on what's called Byzantine fault tolerance the basic idea is that instead of modeling all the ways that individual machines can crash or drop packets or sudden random packets you can design protocols that will work even if some of the machines were controlled by a malicious adversary who does the absolute worst thing at each moment so a lot of distributed file systems are based on protocols that are designed to be robust to an adversary who can crash a machine at the absolute worst time or drop the absolute worst packet or send the absolute worst face fake packet and that's how we have our modern internet economy I think we can possibly get a similar result for machine learning using adversarial techniques to design and verify systems to be robust we're already starting to see a little bit of this there's a technique called relu Plex that was originally created to verify that machine learning algorithms are robust to adversary examples that it's been used to analyze and verify air traffic control systems another really important research area is the label efficiency of our models today supervised learning works really well but you need to have a very large amount of labeled examples and a lot of the time that's just not feasible it can be expensive to gather the labeled examples it can also have other kinds of costs besides money for example if you're working in healthcare to label some of your examples you may need to have patients undergo painful or invasive and risky tests to find out what the true state of your patient is for the machine learning model to predict so it's important to be able to bring down the number of labels that we need one approach using adversarial machine learning to improve label efficiency is to actually take a ganda scrim inator and train it as a classifier instead of just a discriminator when we use a discriminator in the Gant framework it really only has two outputs you can say that the input is real or I can say that the input is fake in the semi-supervised learning framework we can actually extend it to have as many outputs as we want our eventual classifier to have and then we give it one extra input saying that the input is fake so for example if we want to train a model to recognize cats and dogs with very limited label data we can make a discriminator that has three output classes real cat real dog and fake it can then learn from three different sources of data one source of data is fake data coming from the generator and we train the discriminator to say that that data is fake another source of data is labeled real data like we would usually use the train a classifier and there as with a regular classifier we just train it to output the correct label if we see a cat we say output real cat if we see a dog we say output real dog and then there's a third class of data that we can learn from which is really important we can actually learn from unlabeled real data and when we see unlabeled real data we don't know whether it's a cat or a dog but we can just maximize the sum of the probabilities for all of the real classes so we can say we want to increase the logarithm of probability for classification add a domain recognizer network that examines the features the domain recognizer network examines the features and tries to predict which domain that came from so for example you could train on Viper and PRI D and the domain recognizer tries to tell whether the features were extracted from Viper or extracted from PRI D then the feature extractor learns separately in this adversarial game it tries both to learn features that are good for classification but also to learn features that confuse the domain so that all the domains become equally probable this works well if you want to train on viper and PR ID and then deploy to cuhk another really similar idea is called professor forcing professor forcing is designed to solve a problem with generating samples using recurrent Nets when you train recurrent networks on for example texts you feed in a full sentence of words and condition the recurrent net on each word in the sentence so when it's estimating the probability distribution over word I its conditioned on real words one through I minus one so it only ever sees real data when it's guessing the next word it never gets its own outputs fed back in as input for the next time step after it's been trained that way when you deploy it to actually generate new texts it will generate the first word and then it gets conditioned on the first word to produce the second word so it starts to see its own output as input and if the output from the recurrent net doesn't match the distribution of real words exactly then it starts to wander further and further away from valid sentences early on it might pick a few strange words and then it becomes even more strange because it's now producing where it's given unusual inputs professor forcing is an idea that essentially applies the domain adversarial network idea to recurrent Nets where one domain is the recurrent net clamped to real data and the other domain is the recurrent net in freerunning mode we use an adversarial critic to make sure that the statistics of the recurrent Nets hidden units are the same whether it's run on real data or run in free sampling mode one of the most popular domains that we want to bridge the gap between is simulated data and real data one way that we've seen this work is actually using again to enhance synthetic data and make it look more realistic this is useful because if you want to learn to tell where a person's eyes are looking it's actually hard to gather real data with labels about where the eyes are pointed but it's easy to gather lots of unlabeled pictures of eyes with no Direction associated with them it's also easy to make a 3d model and render it with the eyes pointed in directions that you know where they are because you positioned the 3d model yourself if you just train a model on the synthetic eyes though it won't work very well in real photos so what you can do instead is train a conditional again that takes the eyes and makes them look more realistic so then you can take synthetic 3d rendered eyes with known direction and refine them to appear realistic and now you have both properties in one sample you have both realism and a labelled eye gaze direction and you can then train your model to deploy in the real world so that's one way of bridging the gap between head synthetic and real we saw another example of more or less the same strategy around the same time in Google's grasp again the idea behind grasp gown is to train a robot arm to solve grasping tasks where it can pick up objects in this tray grasp again uses synthetic data rendered from a simulator but then uses again to turn the synthetic images to look realistic the value of this data is very high because we get both a per pixel class segmentation mask shown on the right and the relatively realistic image as shown in the middle if we just use real photos we wouldn't have the per class segmentation mask unless we had people label them explicitly and if we just use synthetic data we wouldn't get the realistic appearance grasped gown is able to actually perform better on real data using synthetic and real training data put together than if it could use only real data or using only simulated data to train a more recent approach to generalizing from synthetic to real data is called sim to real via sim to sim the model is trained to deal with many different simulated conditions and as long as there's enough diversity and those simulated conditions it can then deal with the real world in particular again is trained to map from randomized simulated domains to canonical simulated domains so by randomized simulated domains I mean images generated with the simulator configured to use really weird textures like shown in the upper left and then by canonical sim I mean images like the one in the upper right where the simulator is set to use relatively flat textures with one specific color for each part of the robot arm after the gun has been trained to convert images from very many different randomized simulated domains to the canonical simulator we can then use it to process the real world and turn the real world into images that look like they came from canonical simulation as shown in the bottom row because we now have this conversion layer that can turn any image into a canonical image it's possible to train the robot policy only on the canonical simulated images and then when we deploy it in the real world the policy never actually sees a real image it only sees canonical images that were converted on their way and from the camera and it's actually able to learn to grasp without ever using any real data you can of course also improve its performance by using real data in the training it's also important to study fairness accountability and transparency one approach to fairness is if you want to make sure your model ignores a certain variable you can use a technique similar to domain adversarial learning where instead of trying to make it impossible to recover the domain from the features you make it impossible to recover a sensitive variable from the features there are multiple definitions of fairness and this approach doesn't necessarily satisfy all of them it also Montgomeryville guarantee that you've converged to a Nash equilibrium or the sensitive variable has been entirely removed but you can more or less encourage representations to ignore variables that you think should not affect a decision a lot of the machine learning literature has recently focused on interpretability and part of the reason this is important is that we want to go to tell how machine learning systems are working so that we can make sure that they're fair I think unfortunately a lot of the interpretability techniques we have today imply that the systems work better than they do and I hope that we see more talk between adversarial machine learning people and interpretability people we see a lot of systems that examine modern convolutional nets and say look they classify the dog is a dog because of the dog pixels and I classified a cat as a cat because of the cat pixels so it looks like they're working but then at the same time you can take that same model and turn the Panda into a Gibbon without changing any of the Panda pixels significantly so I hope we can get interpretability tools that are more aware of these kinds of failings I also believe that as we make models more robust they'll become more interpretable we've seen that since about 2015 with some of the first work on adversarial examples that we did at brain the image on the left is a linear model which you might think is interpretable because it's linear in terms of people have really bad intuitions for how linear models behave in high dimensional space everywhere you see a yellow box we have successfully turned the input into a different class so as you read left to right top to bottom the first yellow box is where the nine has become a zero according to the linear model the second yellow box it's a one of the 30 other box it's a two and so on so even though this model is linear it's very hard to convey to a human that it has all of these mistakes baked into it the model on the right is a robust model where when we try to maximize the probability of the zero class we actually get images that look like zeros when we try to maximize the probability the one class we get images that look like ones and so on so even though the model is not linear anymore and might seem like it's harder to understand the way that it behaves under optimization of its input is actually much more consistent with human intuitions I'm not saying that this is anywhere near formal enough to have a real fairness tool but it does give me some optimism that studying robustness will make the interpretability problem easier and then lastly I think that adversary machine learning can actually tell us a lot about ourselves and how our brains work you can think of adversarial examples as bugs in machine learning models and we know that the human brain has a lot of bugs like optical illusions but so far the optical illusions for humans have been different than the optical illusions for machine learning algorithms what if we could start to find some bugs that we have in common well recently some of my collaborators and I found that we could actually make images that fool multiple convolutional nets and if you get enough convolutional nets participating in this game and do you give humans a time limit we can actually make images that full time limited humans the picture on the right of the of the spider is usually correctly classified as a spider but if we make an adversarial perturbation of it as shown on the far right humans with a 200 millisecond time limit will most often classify it as a snake and then if you take a more extreme version of this approach and allow it to make bigger changes we can actually take this cat and make it look like it's a dog even to a human with no time limit we don't know for sure whether the dog is really a dog and we've changed it too much we've changed the true class or whether we've managed to fool even the time limited human these are only really the first results about adversarial machine learning for neuroscience but I think that now that we've seen that there's a little bit of signal here there's a lot more to explore and we can learn a lot about the human brain by finding the similarities and differences between bugs and machine learning and bugs and humans thank you for listening to my talk I think I have a few minutes left for questions thank you very much you know for the talk questions there are microphones do we have any questions yeah yeah I was wondering I've really noticed also actually in your talk that there's quite a difference between how much interest there is in adversarial networks in academia versus in industry there's I see very little use of them in industry do you have any feeling of why this is and how it can be solved a lot of the companies that are working on machine learning and industry are not really doing image generation stuff but companies that do work on image generation like Nvidia and Adobe have been a lot more enthusiastic about them I don't necessarily think of it as a problem to solve just if your business isn't image generation then I don't see a reason to use that particular tool well because you mentioned so many other applications today oh I'd say like most of the things I talked about were things like machine learning security where there's other applications I think for things like model-based optimization then people will need to use adverse sale techniques but maybe not necessarily Gantz there's a lot less work on model-based optimization right now I think partly just that's more complicated and less likely to turn into a product immediately than some other things like classifiers or recurrent Nets okay thank you so I really like the example with the Apple in the mesh bag that confuses the classifiers so you said that the restoral training could help us with examples like that can you please clarify because I can't really imagine again generating an appellee in the measure back well so I guess I'm illustrating the the problem is really hard and it's really far from solved I don't actually know how we're going to solve the Apple in the mesh bag if you wanted to think that you would get there with just adversarial training in that context I'm thinking of adversarial training in the sense of optimizing over actions available to the attacker and a threat model so it's a little bit different than the gun version you can imagine if you want to specify a threat model that includes putting mesh bags around things then using again to represent those actions might be a useful way to tackle it but I think we're very far from actually demonstrating anything like that Thanks time from one more yeah hi generative models some argue ganz in particular have poor evaluation metrics such as F ID or inception score what do you suggest like what avenues you suggest that we think about how we evaluate our generative models and moving forward yeah for a lot of machine learning algorithms evaluation has gotten really hard and the gold standard is really to think about what's the downstream task that you want to use them for if you're hoping to use the gun for semi-supervised learning then you can measure accuracy if the classifier at the end if you want to use the gun to generate imagery that pleases humans you can use human evaluation as the property at the end if you're doing something else then you know pick a metric specific to your application area in a lot of cases it's not that the metrics are necessarily bad it's just that there's more than one property the model can have and you're only going to memorize you're only going to evaluate one of those properties at a time so like likelihood measures how much do you put probability mass on the test set but it doesn't tell whether you've over memorized the training set things like fresshe inception distance can tell you how close you are in terms of moment matching to the training data but it doesn't really tell if you've put any mouse on anything new on the test set a lot of the time people just plot curves where they show trade-offs between more than one metric and that can give you some idea of how it might perform on different downstream tasks but I don't think there's any real perfect solution I think it's mostly that we've moved to more complicated models that have more than one performance characteristic and we can't expect to embed it all in one number let's thank the speaker once more [Applause]

Channel: Steven Van Vaerenbergh

Views: 37,480

Keywords: iclr, 2019, talk

Id: sucqskXRkss

Length: 43min 6sec (2586 seconds)

Published: Sun May 12 2019