[Classic] Generative Adversarial Networks (Paper Explained)

hi there today we'll look at generative adversarial Nets by Ian J good fellow at all so this one is another installment in our series of historical papers that had great impact Ganz nowadays or general generative adversarial Nets back then were sort of this was the starting shot in a long line of research that is still continuing today so I remember when I started my PhD in 2015 Ganz were just about spiking I remember new rips or back then nips in 2016 and every other paper was about Ganz it was there was also this famous schmidhuber Goodfellow moment at the tutorial it was it was a wild time and this is the paper that started it all and the paper is quite well written it's very kind of focused on convincing you that this is a sound method mathematically that it does do you know that it doesn't just do wild things and also it is already quite has a lot of the it has a lot of sort of the modern tricks for Gans already sort of built into it so astounding how how much foresight there was already in this paper but of course Gans have come like a super long way since then and today we'll just go through the paper and look at how it looked back then and what this paper was like so yeah join me in this if you like it please share it out let me know in the comments what you think of historic paper reviews this is not going to be like in a beginners tutorial in Gans this is really going to be we'll go through the paper you see right here the paper is from 2014 so it would still be another like two years or so until Gans really take off from this point on but the introduction of course was really important okay so abstract here we go we propose a framework for estimating generative models via an adversarial process in which we simultaneously train two models a generative model G that captures the data distribution and a discriminative model D that estimates the probability that a sample came from the training data rather than G okay this was sort of a new thing now I know I know people disagree with this being a new thing but this was a new thing and specifically this this was the first paper that made something like this really work for data so to have a discriminator D and the words generator and discriminator were also introduced in this paper so you train this D model which is the discriminator and the D model basically decides whether or not a given data point comes from data or comes from the fake distribution and then you have a generative model G that is supposed to just create this data X rather and then coming from the database so you want to sample a couple of times from the data and sometimes you sample from this model G and then the discriminator is supposed to decide whether or not it comes from the dataset or from your count from your counterfeiter like from it is generator G and it's supposed to see it say whether its data fake so you train the D model as a simple image classifier so people already knew how to build image classifiers this was this was shortly as you can see before ResNet came on the scene so people already kind of knew how to build cnn's build really good image classifiers and the thought here was really generative models weren't really a thing until then so people were in language models work too vac was kind of coming up but they were would still be doing like RN ends using these word to vac vectors for generating language in images distant AK generative models weren't really much of a thing so you you would do like compositional models or you would do auto-encoders which were just either really blurry or really really are the factory and there are also approaches like deep belief networks and so on but they have their own problems so there wasn't really a satisfactory way to do image generation that resulted in here really high quality images and now here I think the entire fault and this is not really spelled out but the entire thought here is that hey we know how to train really really good image classifiers right this has been evident in these since since alex net so for two years this was evident how to build really good image classifiers and the question here is to say that rather than also building really good generators can't we like harness the power of building really good classifiers for training and generator right and this this is this idea right here this wasn't the one before as you know in like an autoencoder what you do is you input a sample into some kind of auto bottleneck thing whatever and then at the end you train your output sample to match the input sample as close as possible and then in here after you've trained this this part here is your generative model and then here in here you'd input like MCMC sampling or whatnot and then of course variational autoencoders came up and so on but still what you always would do is you would somehow use the data directly so this is data in order to train your model so you would somehow say ah the output here should probably match the input in some way or in some at least distributional way right this this was a new thing as you can see right here there is no direct connection between the data and the generator and I think this this was the success of this model the fact that the generator did not it wasn't trained from the data like you would do if you were just approaching this problem but the philosophy here is let's use the power of discriminative models which we know how to build in order to train this generator right so the generators task now isn't to match any sort of data point the generators task is to produce images that the discriminator would classify as data and you can do that by simply back propagating through the discriminator to the generator okay so I think that's that's the only thing that's kind of unstated in this paper the the reasoning behind why this is new why this might work but everything else is spelled out very well in this paper I have to say if you read through it so the training procedure for G is to maximize the probability of D making a mistake this framework corresponds to a minimax two-player game so as I said the paper is very much focused on convincing you that there's something sound happening here because at that time if you were to look at this you'd say something like there is no way right you would be like yeah so so I can understand the motivation here to really convince people that you know something something good is happening also on the on the theoretical side in the spaces are in the space of arbitrary functions G and D a unique solution exists with G recovering the training data distribution D equals to 1/2 everywhere in the case where G and D are defined by multi-layer perceptrons the entire system can be trained with backpropagation there is no need for any mark of canes or unrolled approximate inference networks during either training or generation of samples ok so the point here is that it's much easier than current methods of producing of generative models and also it does something sound now let's jump into the loss function right here so they say G and D play the following two-player minimax game with value function V and this is you know still understood until today that it was already like if this was a pure engineering paper they could simply build the architecture and say oh we let these networks fight and they they are kind of adversarial and they they pump each other up and so on and and this here was more much more into the direction of kind of a a theoretical reasoning into why something like this would work of course there is still a lot of engineering going on to actually make it work so they they have there is this value function right here okay and the value function is the following so what you have is you have the log probability of data and you have one the log 1 minus D of the generated samples so here you can see and this was introduced this seems also obvious now right but you have a prior on what this is called the noise distribution okay they have a prior on your input noise to the generator because the generator is supposed to come up with very many different data points and if it is a if it is a you know non stochastic function like a neural network then you need some way to make to produce different images so there is this prior distribution over the noise you feed that noise into the generator the generator will produce an output you put that into the discriminator and then this right here as you can see the discriminator is trying to maximize this objective so the discriminator is trying to maximize the probability of real data and it is trying to minimize the probability of fake data okay it is this is simply a two-way classification problem at the same time the generator as you can see is trying to minimize the objective in fact the order here is quite important so the generator as you can see is trying to minimize whatever this here is so the generator is sort of is trying to minimize against the best possible discriminator and so this is one one observation right here is that the formulation is always with respect to a perfect discriminator now we know that this doesn't work with if you have a perfect discriminator than generator cannot catch up because you have insufficient gradients and so on and this was already recognized in this paper as well but the formulation is with respect to a min Max game and not a max min game so the other point I want to make here is that you can see the discriminator appears in both in both terms right here however the generator only appears right here okay and this this basically means that the objective for the generator is only this part here because the other part is constant so the generator is just trying to make the discriminator think that fake data is real so it is trying to make the discriminator the class of fake data as small as possible for the data that it outputs while the discriminator is trying to make the class of fake data more than the class of sorry real data yeah it's trying to make it it's trying to classify fake data as fake and real data as real whereas the generator as only this part on the right this is I feel this is um it's quite important um why because already in this paper they recognize that this might not be the best practical objective and for the generator they can actually exchange this part here on the right to simply say we want to so we want to instead of 1 minus D instead of log 1 minus D we simply want to use minus log D as an objective for the generator so you can kind of play around with this and as you know lots of formulations have played around with this loss right here and yeah that's why we have like a billion billion billion billion gam variations the introduce the reasoning behind this so that there's an intuition right here and you can see already in practice equation one may not provide sufficient gradient for GE to learn well early in learning when G is poor D can reject samples with high confidence because they are clearly different from the training data in this case this saturates rather than training G to minimize that we can train G to maximize log D this objective function results in the same fixed point for the dynamic but provides much stronger gradients in early much stronger gradients early in learning this is in contrast to like other papers that simply say oh we do this and they at least say it provides the same fixed point right yeah so again they're trying to convince you that this is doing something useful and that this is easier okay so this strategy is analogous to other things training maintain samples from a Markov chain from one learning step in the next two order to avoid burning in the Markov chain in another loop of learning sorry okay this is from another paper so their point here is that it's analogous to other papers that use these markov chains where you always do one step in GE and one step in d we alternate between K steps of optimizing D and one step of optimizing G because you have this inner maximization over D and then the outer maxim is it in the outer minimization over G so this has already been around the fact that you kind of have to have these optimizations in lockstep but the difference here is you don't need any sort of like Markov chain in the inner loop and so on you simply need back propagation so here's an illustration of how that might work so at the beginning here you have your z space and this is always sampled uniformly as you can see right here this is from a prior distribution and through the mapping so this here is from Z to X is G so this is the mapping G you can see that the uniform distribution is now mapped to something non uniform which results in the green thing so G is the Greenline while as this is data the black dots are data and if you have a discriminator the discriminator is supposed to tell you where there's data and where there's fake data now so green here is fake now this blue line is sort of a half-trained discriminator now you train d right you max maximize d the discriminator and that gives you this blue line right here so this this is a perfect discriminator for these two data distributions it tells you it's basically the the ratio of green to black at each point and now you train the generator according to this and you can see that the gradient of the discriminator yes so the gradient of the discriminator is in this direction okay so it's like up this hill and that's why you want to shift your green curve over here according to the gradient of the discriminator note that you know we first trained the discriminator and now in the second step we mean we optimize the generator so now we shift this green curve over in order to in along the gradient of the blue curve so it's important the green curve doesn't see the black curve ever the generator doesn't see the data the generator simply sees that blue curve and it goes along the gradient of that blue curve of the discriminator ok and then if you do this many many steps actually there are dots right here you will end up with a discriminator that has no clue what's where this is 1/2 probability everywhere because the ratio is the same and you end up with the probability of data equal to the probability of the output generated samples and this can happen if the generator simply remembers the training data but there are a number of things that counter that for example the generator is continuous while the training data is of course a discrete so there is this in between things right here where there is no training data in fact you hit a exactly training data is very very unlikely but of course you can still you can still peek at the training data but also the there I think there are two things while the generator doesn't simply remember the training data first because it doesn't ever see the training data directly so it can only see it through the discriminator and second of all because it is built as these multi-layer neural networks it doesn't have the power to just remember this because as there is kind of this notion of continuous function so and the these neural networks are rather smooth functions often and therefore I think that is something that helps the generator avoid remembering the training data of course there is still this problem of mode collapse that was really big in Gantz so even if it doesn't remember the training data it might focus on the easiest part of the training data and forget all other parts and that was a direct result actually of this objective so where it was it so this objective directly led to mode collapse in some in some form because it penalizes different errors differently so of course people have come up with ways to to solve that okay now here is the algorithm as you can see this was already quite it was already quite the algorithm we use nowadays so for K steps this is the inner maximization and here they say that we use K equals 1 so all this is this is pretty much what we use today the early days of Gann were still like how much do I need to discriminator per generator and so on nowadays everyone's just using one step here one step there or even training and jointly works in some cases so you want to sample a mini batch of noise samples and you will sample a mini batch of em examples from training data generation so from this data you want to update the discriminator by ascending it's stochastic gradient and this is simply the gradient of the objective and then after those K steps you're going to sample another mini batch of noise samples and update the generator by descending it's stochastic gradient and you can see right here already there is this reduced objective that doesn't include this because it falls away in the gradient right and they say the gradient based updates can use any standard learning based rule we use momentum in our experiments very cool so I believe they already also say that it is somewhere here it's pretty it's pretty fun that they say oh in our generator we only input noise at the lowest layer this is also something that if you think that G here is a multi-layer network so it's kind of a multi-layer network that outputs an image right and if you ask yourself if I have noise how would I input that into there it's so clear nowadays that you know we just put it here but this was not clear at all this was kind of an invention of this paper because you could you know put it pretty much at all layers you could distribute it and so on you could add some right here it it was this paper that already established the fact that we input noise kind of as a vector at the very beginning and then just let the neural network produce the image from that so yeah pretty pretty cool it's pretty sneaky how many things are hidden in these initial papers how many decisions that are made there then are just taken over and you know this one I guess turned out to be fairly fairly good okay so here they go for some theoretical analysis and the first they want to convince you that if the the generator if this all works well if this if both parties this generator and the discriminator optimized their objective to the optimum then the generator will have captured the data distribution so the global optimality of this and they go about convincing you of that so the first thing that they convince you of is that if you fix the generator the optimal discriminator is this and we've already seen this in this drawing right here so the optimal discriminator is simply the ratio of the data of the likelihood of data versus the likelihood of the generated data okay so you train you're always trained eat this discriminator in the inner loop and that's simply the consequence of this of a point wise this is true point wise therefore it's true over the entire data distribution in the next thing they convince you that the global minimum of the virtual training criterion and this is the value function this min max game is achieved if and only if this holds at that point the training criterion achieves the value of negative log four and this again this was already already here the fact that this has a global minimum and it is achieved when the generator matches the data distribution which is pretty cool so in the proof it's pretty simple actually they first say look if this is the case we just simply plug that in this the discriminator will be confused so if the generator exactly captures the data the discriminator will have no clue what's going on right because it can't because they're equal so it must basically output the probability of 1/2 everywhere and then your objective becomes a constant negative log for now if you then plug that into the other equation you'll see that the training criterion ends being negative log four plus twice the Jensen Shannon divergence between the data and the generated distribution and since this term here is always positive that means that this thing here can never be less than negative log four and therefore the negative log four is the optimum okay that's it's that the proof is is pretty cool I have to say to show that this has the optimum at that place and the last thing they convinced you of is that this algorithm actually converges and the converges is simply predicated on the fact that if you look at each of these problems individually they are convex so like here is convex and X for every alpha so each of these are sort of convex problems and then it will naturally converge to the two their minimum however in practice adversarial nets represent a limited family of distributions via the function and we optimize the parameters rather than the distribution itself using a multi-layer perceptron to define G introduces multiple critical points in parameter space however the excellent performance of the multi-layer perceptrons in practice suggests that they are a reasonable model to use despite their lack of theoretical guarantees so they say if we could optimize this probability distribution directly it is a convex problem and we always converge but in practice of course we only optimize the parameters of an MLP or a CNN and that doesn't always converge but we have reasonable hopes that it will converge okay so again it's very much focused on convincing me that this is doing something sensible which I hope now you are convinced so there is a global optimum point it's when the generator captures the data distribution perfectly this is this can be achieved and we we'll be achieved if you can optimize these probability distributions with a reasonable degree of freedom and the neural networks provide that reasonable degree of freedom and you know give us good hope that in practice it will work so they apply this to data sets namely M nest the Toronto phase database and C 410 the generator Nets use the mixture of rectified linear activations and sigmoid activation z' while the discriminator net used maxout activations that was still a thing dropout was applied in training and the discriminator net while our theoretical framework misused other data yeah while our theoretical framework permits the use of dropout and other noise at intermediate layers of the generator we used noise as the input to only the bottom most layer of the generator Network again this wasn't kind of clear at the beginning and also the fact that to leave out dropout and so on in the generator was I guess they found that empirically and then there was of course no way to evaluate these things like how do we evaluate generative models nowadays we have these inception distances and so on but then we estimate probability of the test set under P on the regenerated data by fitting a Gaussian parson window to the samples generated with G and reporting the log-likelihood under this distribution the theta parameter yada-yada results are reported this method of estimating the likelihood has somewhat high variance and does not perform well in high dimensional spaces but it is the best method available to our knowledge advances in generative models that can sample but not estimate likelihood directly motivated further research into into how to evaluate such models they were absolutely right in this and there was a lot of research into Europe into how to evaluate these models however I it is my opinion that we still have very very limited methods of evaluating models like this like we have better methods but it's yeah it's not really it's not really satisfactory how it is right now so you see that these models these adversarial Nets by the way they're always called adversarial Nets right here well I think we call them like most people would call them adversarial networks but it's just interesting to see the nets and also in the title right it says I think it says Nets does it I think it does we'll look at it after so the T out they outperform these other models in especially these these belief networks were kind of popular at the time and you can see the samples right here were in no way comparable to examples that you get from the modern Ganz but this was already very very very good especially the emne stand then here you could ask actually recognize so the once what the yellow are always from the training data set they're like the nearest neighbors of the things on the left so they want to show that it doesn't some simply remember the training data though I'm not so sure like this seems like it has some surge somehow remember the training data a little bit also this one right here and there was already a way so this was also very farsighted so these a to see were fully connected networks which might be one of the reasons why worked moderately well right but the last one was a convolutional discriminator and AD convolutional a generator so already using kind of d convolutions that are used everywhere today so they are used in in ganz in whatnot we VA is to upsample anything if you want to do pixel wise classification you use d convolutions so again this this paper sort of introduced a lot of things that later that we still use in guns today now I'm sure D convolutions weren't invented here but you know we still we still use them so legit they were the first gaen paper to use the convolutions haha yeah they also say we make no claim that these samples are better than samples generated by existing methods we believe that these samples are at least competitive with the better generative models in the literature and highlight the potential of the adversary framework today this paper would be so rejected like hey wait you're not better get out of here you can't claim you can't claim this anymore doesn't work anymore I'm sorry yours has always has to be better than everything else nowadays otherwise it's a it's it's a weak rejecter experimental evidence doesn't doesn't convince me you can't simply say something's cool also already introduced in this paper digits obtained by linearly interpolating between coordinates in z space of the full more like this thing here every single gantt paper had interpolations in the like in this in the ganz bike and it came all came from here so already this is just like this is a like every gam paper then had like rows of these like of these interpolations and I should know I've I've written the paper on it and introduced right here who knows if they hadn't done this yeah I guess it's it's kind of an obvious thing but still you know very very cool to see that this was already done and here ganz compared to other different methods like deep direct graphical models generative auto-encoders and compared in very many ways so this is a actually good reference if you want to learn about these different kinds of models and they make the claim here that there are advantages and disadvantages so disadvantages mainly come with training these things because you have to train them in lockstep but then also the disadvantage is that you don't have an explicit representation so there there is no explicit representation of this probability distribution you never build the data distribution you can only sample from it however the advantages are that Markov chains are never needed only back prop is used to obtain gradients no inference is needed during learning and a wide variety of functions can be incorporated into the model this you know I hadn't read this paper in a while and I just have to laugh nowadays because you know now all the people are trying to reintroduce like there are as many papers like reintroducing Markov chains into Gans being like oh ma Gans would be so much better if they had an MC MC sampler somewhere you're like no this it the point was to get rid of it and like no inference is needed during learning which you know for some of these other models you actually need an inference during training right so this is very very costly and how many models are there nowadays where it's like oh if we just do this inference during training yeah so it it's quite it's quite funny to see people kind of trying to to just combine everything with everything and in the process sort of reverse reverse whatever these methods were originally meant to get rid of now I'm not saying anything against these methods but it's just kind of funny yeah so they had a lot of conclusions and future work they already say you know conditional Gans are very easy to do straightforward learned approximate inference can be performed by training an auxiliary network to predict Z given X and this of course as you know has come you know it has come to fruit very often early papers already introduced the D so if you have the G network producing some producing an X and then the D network discriminating that you would also have like a encoder right here to produce back the Z noise to give you the latent encoding sort of like a variational encoder but not really it's more like a reverse generator you know this models nowadays are big by Gann and things like this that employ this exact thing that was sort of predicted right here of course there are much earlier models also using this as long as I can remember people have attempted to bring encoders into Gans easy they have a bunch of other things like semi-supervised learning you can use this to do to do get more data for a classifier which is also done so a lot of things here already foresight in this papers it's pretty cool and the coolest thing look at that savages Goodfellow not even using the full eight pages just not dropping this on the world absolutely cool mad respect yeah so yeah this was kind of my take on general yeah it is generative adversarial Nets and yeah you'd please tell me if you like historic paper overviews it's more kind of a rant than it really is a paper explanation but I do enjoy going through this papers and kind of looking at them in hindsight all right that was it for me I wish you nice day bye bye

Channel: Yannic Kilcher

Views: 39,619

Keywords: deep learning, machine learning, arxiv, explained, neural networks, ai, artificial intelligence, paper, gan, generator, discriminator, convolution, deconvolution, goodfellow, bengio, convolutional neural network, mnist, cifar10, generative, generative model, image generation, face model, latent space, interpolation, minmax, nash equilibrium, game theory

Id: eyxmSmjmNS0

Length: 37min 4sec (2224 seconds)

Published: Sun Jul 19 2020