Build a Generative Adversarial Neural Network with Tensorflow and Python | Deep Learning Projects

one of the most interesting parts of machine learning is generative machine learning in this video we're going to be using that exact technique specifically a generative adversarial neural network to get started with generative machine learning let's do it [Music] [Music] [Music] so in this video we're going to be covering a bunch of stuff but specifically we are going to be focusing on generative machine learning using a generative adversarial neural network now we're going to be using this to build and design our very own fashion line but this could be just about anything you could generate synthetic images of a whole range of different types of fields or spheres or objects who knows now we're going to take this step by step and code it up all the way from scratch so this is going to be installing our dependencies getting our data visualizing our data then we're going to get to the good bit and this is where we actually build a generator model so this is the actual neural component that is able to generate new images or new objects we'll then take it a little bit further and design our discriminator model think of this as a art critic which is trying to detect a fake from a real image and we'll wrap it all up in a custom training loop and then visualize our results ready to do it let's get to it hey nick what are some use cases for gans well basic gans are great at generating synthetic data more advanced models allow you to do things like super resolution smoke removal and style transfers wait think you could build us a simple game and maybe generate some fashion sure let's get to it first thing we need to do is set up our environment welcome on back to the breakdown board i really wanted to do that it's been a while okay so in this video we're going to be building again but not just any game we're going to be building ghana generate or again to generate fashion so the first thing that we're going to be needing to do is get some images of fashion now in order to do that we're going to leverage the tensorflow data set api aka tfds and we are specifically going to be using the fashion mnist data set so this is going to allow us to get images of things like t-shirts shoes let's get draw a boot or something and things like bags as well so like it's got a whole bunch of random images that we can actually use for our particular data set so the thing that we're then going to do is we're actually going to have to build up a generator so generator is going to take in a bunch of random numbers and this is commonly referred to as a latent dimension so it might be like 1 23 42 71 so on and what that is actually going to output is it's going to take in those random numbers and it's going to output a set of values which represent an image so what we're actually going to do so let's say for example the images that we're getting back from tfds are going to be in the shape 28 by 28 by 1. we might take in some random values into our generator let's say in our particular case we'll take in 128 random values those 128 values will be reshaped into a format that allows us to output a set of values which are 28 by 28 by 1. so we'll actually take those 128 values and we'll build up our generator so our generator will be comprised of a number of convolutional neural network layers as well as some up sampling layers and eventually what we are going to get probably connect these hey we're going to output an image which is 28 by 28 by one which is effectively a generated image that is the crux of again now this is where the discriminator comes in we'll actually take the generated image and pass it through a discriminator neural network and that is again going to have a number of convolutional neural network layers but the final output for that neural network is going to be a zero or a one which represents whether or not it's a true or a fake image so we've set it up so that zero represents true and 1 represents fake now other implementations might do this a little bit differently but this is definitely going to work so we'll then take that generated image and bang it into our discriminator at the same time we'll also take real images all the way from tfds i'm not going to let's just make an uh entry so one will be going here so that will come from our tfts model and we'll be able to train our discriminator as to what is real and what is true now this is where the fun bit comes in the custom training loop i've drawn uh some dude lifting weights here but you kind of get the idea we actually need to balance off the training between our generator and our discriminator and this is a key step when it comes to actually building guns we actually train a discriminator and it gets rewarded if it effectively picks out fakes whereas our generator gets rewarded if it's able to fool the generator or fool the discriminate up so generator its goal is to fool the discriminator now the cool thing about keras and tensorflow which is going to be the deep learning framework that we're actually going to use is that we can actually create a custom train step and through here we can actually build that custom training loop which allows us to build a generator now finally when it comes to actually testing this model we can actually take the generator model specifically discard the discriminator because we don't really need it anymore it's purely needed for training we can then pass through those 128 random variables to our generator let's say that this is our generator is represented as a pink model can mod and then what that is going to do is it's going to output our different images of fashion so we might get a t-shirt we might get a boot so on and so forth now there's a whole bunch of applications for this again you can use it for generating other types of stuff you can even go to a slightly more advanced model called a conditional gan which gives you a little bit more control as to what it is that you actually generate for now we're going to keep it simple let's get to it alrighty so the first thing that we told our client that we were going to do in order to build up again is set up our environment now in order to go through and complete our game there's a bunch of stuff that we're going to do but we'll set up our environment first so once we've done that we're going to visualize and build our data set we're gonna take a look at building our neural network so we'll build up our generator our discriminator which i'll talk about more lately or later and then we're gonna construct our training loop which is pretty involved but we'll walk through it and then we'll test out our generator but first things first let's go on ahead and set up our environment now the first thing that we need to do is install some dependencies so the main dependencies that we're going to have around here are tensorflow matplotlib so let's go ahead and install those and then we'll be able to kick things off okay so those are all of our dependencies that we're going to need so i've written one line of code there and the full line is exclamation mark pip install tensorflow tensorflow gpu map plot lib tensorflow dash data sets and ipi widgets so tensorflow and tensorflow gpu are going to be the core dependencies that we're going to need to actually build up our deep neural network matplotlib is largely going to be used in the visualization component so when we actually go and visualize our data sets in the next step after we've had a chat to our client we're going to be using that tensorflow datasets is the actual data set that we're actually going to be using so if you go to tensorflow data sets and actually take a look so there's a bunch of data sets available now we're just going to be using the fashion data set so i think we can go into the catalog and search for fashion i think it's in here yeah so we're going to be using fashion mnist so there's a bunch of images from zalando's article images so they're all 28 by 28 grayscale so this is just going to make building our deep neural network a little bit faster rather than using monster images like i've done for the super resolution tutorial which will be coming around kind of soonish cool all right so those are data sets and then ipi widgets is just a dependency whenever you're downloading tensorflow data sets if we go and run that i'm pretty sure i've got all of these installed already so it should run relatively quickly and it has let's double check we've got no errors so we've got a warning saying we need to upgrade pip that's perfectly fine we can ignore that now i always get the comments nick show us what versions you're using so i'm just going to run a quick pip list and show you what versions of stuff i'm using so the core things that we need to take a look at are tense flow so over here you can see i'm using tensorflow 2.8.0 and tensorflow gpu 2.8.0 tensorflow data sets we're using 4.5.2 what else did we have we installed matplotlib let's take a look at that matplotlib we're using 3.5.1 so that should hopefully get you across the board if you're trying to work out what versions of stuff to install keep in mind that the versions of tensorflow that i'm using are namely because i'm matching the version of cuda and cu dnn that i've got installed so if you i've probably shown this a billion times but i want to show you just once more so if you actually type in tensorflow gpu into google and you actually take a look at use gpu uh there should be i'm not sure if it's on this page but there's actually a no it's not on this page so there's actually a compatibility guide here we go it's down here so over here on the left hand side build from source actually shows you so if you scroll all the way down this actually gives you the the configurations that you're going to need so i'm using 2.8 2.9 is now out um so i'm using 2.8 which means that i need to have and i'm running this on windows which means that i need to have microsoft visual studio code 2019 bazel 4.2.1 and then these are the two most important bits in order to enable gpu acceleration we need cu dnn 8.1 or ku and kuda 11.2 not optional you need both just keep key thing to keep in mind as to why i use specific versions okay so that is our set of dependencies now installed the next thing that we need to do is first up what we are going to do is limit the gpu consumption so if i actually show you my little gpu at the moment so right now you can see that we've kind of spun up some stuff so we might need to shut down some existing kernels but what we want to do is we want to limit the virtual ram growth and this is a key thing whenever using tensorflow it'll expand to use all of that ram so you want to constrain that a little bit so you don't get a ton of out of memory errors so we can do this using uh the gpu i think it's set memory growth equal to true so i'm going to show you how to write this command and then we'll take a look so let's do it okay that is our memory growth now set so we've gone and written four lines of code and you've probably seen me bring this one in a ton of times before so the first line that we've written is import tense flow as tf so this is going to allow us to leverage tensorflow so uh bringing in tense flow and then if you've ever watched any of my other deep learning tutorials these three lines are going to be super common so first up we go and grab all of our gpus and the full line there is gpus equals tf dot config.experimental.list underscore physical underscore devices and then we're going to pick up all of our gpus so if i actually type in gpus you can see i've only got one there so i've got a 2070 super on this deep learning machine and then what we're doing is we are looping through each one of those so if we actually did have multiple then we'd actually go and loop through every single one of them this just sort of sets you up for scaling if you were to use multiple gpus later on so the next line is for gpu in gpu so it's effectively looping through this so for gpu and gpus print gpu so you can see it's going to print out each physical device then we're actually setting that memory growth so remember i sort of mentioned that that function so tf.config.experimental.set underscore memory underscore growth we're passing our gpu so that particular gpu is part of our iterator and we're setting that growth equal to true so that means we're not going to blow out our vram when it comes to actually building up our deep neural network now the last thing or the second last thing that we need to do is actually import the rest of our dependencies so we've so far installed a bunch we've taken a look at our versions and we've limited our memory growth we now need to bring in the rest of our dependencies and then bring in our data so first up let's bring in the rest of our dependencies okay those are our next two dependencies now brought in so i've gone and written two lines there so the first thing that we're doing is bringing in tense flow data sets or fashion m nist so that line there is import tensorflow underscore data sets as tfds so this is going to allow us to use this fashion mnist data set and so we're going to effectively be able to generate images that kind of look like this which could be scaled out to be able to generate color images could be scatter to generate bigger images there's a ton of stuff that you could do so we are bringing in tensorflow data sets and then the next line that we're bringing in is bringing in matplotlib map plot lib for vis stuff so the line there is from matplotlib import pi plot as plt so really just two core lines of code there to bring in our dependencies now the next thing that we need to do is actually download and import our data set so let's actually do that okay and that is our data set now loaded so we've actually gone and used the tensorflow data sets api so use the tensorflow data sets api to bring in the data source so we've written one line of code there so it's ds equals tfds.load so you can see that there so tfds.load and then we've specified what dataset we actually want to bring in which in this case is fashion mnist so it's this one over here and the cool thing about the tensorflow data sets catalog is that you can kind of just reference it as that and bring in a ton of different types of data sets and i think i've talked a little bit about how to work with the tensorflow data set api previously not getting it from the tensorflow data sets but building your own but this is how we go and download a pre-existing one so tfds.load is your friend and then you pass through which data set you want so let's say for example you wanted i don't know images of satellites you could actually pass through eurosat and you would get satellite images it doesn't actually show up there but that's sort of the cool idea behind it that you can iterate really quick and then we specify that we only want the training partition so if specified split equals train so the full line is ds equals tfds.load and then we're passing through fashion mnist as the first uh positional argument which is just a string and then we're passing through a keyword argument to split equals train so we now actually have our data set so if we actually went and took a look so ds dot as numpy iterator let's take actually let's take a look at the type first okay so this if we take a look this looks like it's a prefetch component so it's part of our tensorflow datasets api so if we went and typed in dot as numpy iterator dot next we should be able to get a array there you go so what we're actually getting back is a dictionary so you can see the first component is a the image itself and then if we scroll on down let's actually take a look at the keys so we've got an image and then we've got a label so if we actually went and grabbed the image right so this is just an image so we're actually going to take a look at how to visualize this in a sec but just know for now that we've got an image and this is obviously a classification data set so we can take a look at the labels as well in that particular case alrighty that is the first component now done so we've now gone and imported our dependencies in our data so remember we went and installed a bunch of stuff specifically tensorflow tensorflow gpu matplotlib tensorflow data sets and ipi widgets we then went and limited our memory growth we went and brought in a couple of dependencies so tensorflow data sets and map plot lib and then we went and downloaded our data set that we're going to use for again from the tensorflow data sets catalog and we're bringing fashion mnists to be able to generate our own fashion components and then we also took a look at ad briefly look at our data set but we're going to dive into that a bit more in a second all right let's jump back on over to our client and have a chat so that's all done it's done well what next it's always good practice to visualize our data in this case we've got some pretty small images because it's just a prototype but it's still worthwhile taking a look with matplotlib nonetheless so as i mentioned it's good practice to visualize your data set before you try to do anything with it because otherwise you're sort of just walking in the dark which can be a little bit problematic when you're building machine learning things so what we're going to do now is we're actually going to take a look and visualize our data set now in order to do that first thing that we're going to do is bring in numpy so import numpy as np and we're specifically going to use numpy to do do some data transformation which we'll see in a second so now that we've actually gone and grabbed or brought in numpy the next thing that we actually want to do is build an iterator so you would have seen here before i was actually creating a loop to be able to continuously grab data out of our tensorflow datasets pipeline so what we've actually got here so by downloading our tensorflow or fashion mnist data set using the tensorflow data set source we've actually got a pipeline don't think of it as you've loaded in all of your images into memory we've got a set of repeatable calls that we can make to this pipeline to bring data back right now in order to bring that data back we first up need to actually set up an iterator so this is think of this like almost like a connection right we can then call next to bring back the next batch of data so by setting up the iterator and then going next it'll bring back one batch or one image depending on how big the batch size is then if we go next again and bring back another one next again it'll continuously bring back more data so what we're going to do is first up set up an iterator which is effectively just going to look like this right and we're going to call it data iterator set up let's call it connection setup connection aka iterator so then what we can actually do is just call the iterator and type in dot next and this will continuously bring back a batch traditionally you'll batch up the data set so that you've actually got multiple images per batch or multiple samples per batch so if we type in data iterator derator dot next so you can see that that's one image then if i call another one that's actually another image so watch these numbers down here right they're actually going to change so you can see that that changed changed change so we're actually bringing back new data sets each and every time boom boom boom boom boom boom boom so what we're actually doing is we're calling out to that pipeline and bringing back a new batch every single time so it's not loading it all into memory to begin with it's actually making a call and bringing back data as and when we need so this helps preserve some of the memory on your computer particularly when you're doing hardcore deep learning okay so now that we've taken a look at how to get data out of a pipeline so getting data out of the pipeline boom all right and then we can uh cool thing about jupiter lab i don't know if you've seen this is that if you actually right click on an output cell you can actually dis or enable scrolling for output so then rather than me having to go all the way down to find the next cell or go to the next bit it actually allows me to condense the output so if i enable scrolling for outputs you can see that much cleaner now so i can actually see what the hell i'm doing um hopefully that that makes your life a little bit easier as well okay so what have we done brought in numpy set up our iterator aka our data connection and then i'm showing you how to get data out of the pipeline the next thing that we actually want to do is do a little bit of is so the first thing that we're going to do is create some subplots so we're going to use plot.subplots to actually create some subplots using matplotlib and then we'll plot out four images so let's do it okay there we go so we've actually gone and visualized some of our images now so let's take this step by step and let me show you what i've actually written there so i've written one two three five different lines of code so the first line is actually establishing our think of the way the matplotlibs is almost like setting up the format and then you plugging in images particularly when you're working in subplots so first thing that we're doing is we're setting up the format so set up the subplot formatting and all we're really saying here is that we want four columns and we want the total figure which is the total plot to be 20 by 20 pixels in terms of size so then the entire figure is referenced as figure so this is the fig is actually the whole thing ax is each one of these individual subplots right so ax could just as easily be called subplot one subplot two subplot three so if we actually take a look at ax right it's just four different components so that's the first subplot that's the second subplot that's the third that is the fourth right and then we can actually visualize something in each one of those individual little subplots which is exactly what we've done the first thing we're doing is we're setting up the subplots and the return values that we're going to get back are the entire plot the whole thing and the axes which is one two three four because we've specified end calls equals to four then we're looping four times so we're going and grabbing four individual images from our data iterator which is over here so we're just calling dot next four times which is exactly this so for idx in range so let's say we're going to loop four times and get images so for idx in range four first thing that we're going to do is we're going to get a batch now a batch is effectively just a sample in this case i could just as easily call this sample because it's not really i don't believe it's been batched up right so we're getting one sample by calling data iterator.next and remember that sample is going to be comprised of two parts the image which is part of a dictionary and then the label so the first thing that we're doing is we're using the axes and we're using the i am show function which is just the image show function available inside of matplotlib and then we're passing through our image now we're squeezing it because it's actually inside of a set of subarrays so if we actually go and grab an image dot shape so what we actually want to do is we want to condense this down so we've got one a single value down the bottom so if we actually type in np dot squeeze get rid of that so you can see that we're now collapsing it down so if we type in dot shape now it's just 28 by 28 which allows us to visualize it a whole heap easier okay so what are we doing so we're first up getting the axes that we want to visualize so we're going to grab the first index or the first axis so let's actually take a look so ax so we're effectively going all right grab index 0 and then what we're going to do is we're going to type in dot i am show and then pass through the image which is exactly what we've done there so mp.squeeze we're grabbing our sample image and then we're passing it to that so that is what actually does the visualization so let's actually write some comments so grab an image and label plot the image using a specific axis actually we'll call it subplot and then the next thing that we're doing is we're just setting the title this is purely optional like if i commented that out it'd just get rid of the numbers at the top now the numbers at the top are not all that descriptive because we haven't actually gone and taken a look at what each one of those labels are but presumably label 5 is going to be some sort of high heel label 8 is going to be a bag label 6 is going to be a jumper because you can see those two there so this is just printing out or appending the image label as the plot title and to do that we've written ax and then we're passing through the index so we're grabbing this specific subplot at a point in time and then we're using dot title so we're grabbing the dot title attribute and we're using the dot set text function to be able to set the text above that so you can see those little fives let me zoom in on that so you can see that five there you can see that 8 there so on and so forth right so that makes it a little bit easier to see what we're actually plotting out so again we can keep plotting we're going to see different images so at 4 might be a men's jumper 5 might be a shoe one might be some pants nine a boot for a men's jumper nine a boot again three a singlet or something like that uh you sort of get the idea right so we can actually visualize each of those images and because we're using the dot next function we can actually keep getting different sets of images back pretty cool right so we're getting a whole bunch of different images performance jumper nine boot eight bags six what is that singlet jumper whatever you sort of get the idea but we've got our fashion images now we're actually going to use again to be able to generate these types of images now if you think about the capabilities of this we could actually use it to be able to generate synthetic data sets we could use it to be able to generate faces later on because all we'd really need to do is sub in different types of images and use a similar pipeline to do this type of thing okay so that is our visualization now done so so far what we've gone and done is we've done a little bit of data transformation using numpy because remember we used np.squeeze to collapse our array we've taken a look at how we can set up a connection to our data set how we can go and get data out of the pipeline and we've taken a look at how we can go in ahead and visualize it and guys as per usual all this code is going to be available via github so i'll include the link in the description below i'm just going to clean this up a bit alrighty cool now the next thing that we actually need to do is do a little bit of data processing so right now these images are represented as values between 0 and 255 in order to build good deep learning models we typically want to scale values to be between 0 and 1. so we're actually going to set up a quick function that allows us to scale our images so let's go ahead and do that cool so that is our function to scale our images so scale and return images only now we're actually going to transform our data pipeline so that it only returns the image we don't actually need the label at the moment because it's not a super v supervised classification problem if we're going to build a conditional gain which i might talk about in a future video we might actually need those labels because we'd actually want to pass through what type of image we want to generate for this particular model we're just going to be generating different types we're going to sample from the latent space be able to generate different types of fashion so let's take a look at our function first up so first up we've defined a new function and then we've set it equal to scale images so that's going to be the name of it and then that is going to take in the data from our tensorflow data pipeline we're then going to extract the image only because remember the dictionary that we got back included the image and the label we just need the image for this so image equals data and then we're grabbing it using the key image we're then returning the image divided by 255 so this is going to scale our image to be between zero and one hopefully to build a neural network that performs a little bit better and trains a little bit faster cool all right so that is that now what we need to do is actually apply that to our data pipeline so if you've seen uh what was it i think it was actually the image classification video you'll know that i remember how to do this by the initialism mixture app so remember we need to map our data set we need to cache it we need to batch it uh what is it b a and then p and then prefetch so let's actually oh wait there's an s we also need to shuffle it so uh let's actually take a look at how to do this so we'd actually go map so and then cache and then s which is shuffle and then batch and then prefetch so these are the steps that you typically go f-e-t-c-h there we go that's how it is spelled prefetch so these are the steps that we typically do whenever we're going and building up a data pipeline for tensorflow so that is exactly what we're going to go on ahead and do so let's go and do it okay that is our data set now prepared so as i mentioned we are going to run through the mixture back pipeline so map case shuffle batch and prefetch so the first thing that we've gotten done is reloaded the data set completely optional you could just as easily use the data set that we brought in from right up here i've just gone and i like having it in a little condensed area so ds equals tfds.load and again we're loading fashion mnist just so you get used to actually how to do that we're grabbing the train partition all right then this is where the magic happens so first thing that we're doing is we're running the data set through the scale images pre-processing step and i really like the way that this works because we can actually take like almost apply like data pipelines which is effectively what this is doing so we're taking our data and we're running it through the scale images function we could go and do other stuff if we wanted to do data augmentation i've got a better idea of how to do that now we could actually go and apply a data augmentation step here as well but now we don't actually need it then what we're doing is we're cashing our data set so cache the data set for that batch we're then shuffling it so this ensures that we've got a shuffle data set so we're not just looking at a specific set of samples so shuffle it up and so actually let me just quickly walk you through this so first up what we're doing is we're grabbing the data set that we defined up here and then we're overwriting that variable so ds now equals ds.map and we're returning it once it's being passed through the scale images function we're then caching it so we're taking ds and then we're caching it so by applying the cache function so ds.cache we're then shuffling it up so again we're grabbing the ds or the data set we're then passing through or using the shuffle function and we're specifying it as specifying the value to that shuffle function as what the shuffle buffer needs to be which in this case is 60 000 and then we're overriding the variable we're then taking the data set again and we're batching it into batches of 128 images batch into 128 images per sample so now our data set is now equal to ds.batch and then we've passed through 128 today and then we're pre-fetching so this eliminates bottlenecking or slows down reduces the likelihood of bottlenecking so again we're taking our ds variable and we're using prefetch passing through 64 and then that gives us our data set so if we run that no issues so again we can type in ds dot as numpy iterator and now what we should get back when we type in dot next is a set of images which is 128 samples in length which should be 28 by 28 i think by one so if we go and do that type in dot shape so there you go so 128 images because we've gone and batched it into samples of 128 it's 28 pixels what is that wide so i think wide or high 28 by 28 by one right so that's going to be the dimension so i believe it's width and height then channels so it's just a grayscaled image hence why we've got the one at the end there so that is our data set now built so we've gone and done a ton of stuff here so first what we've got and done let me make sure we're zoomed out so we've gone and brought in numpy for a little bit of data transformation when we're visualizing so we're specifically using np.squeeze we're then setting up the connection to the iterator we're getting the data out of the pipeline using the dot next function we're then visualizing using matplotlib subplots we're then scaling it so this is how you define a pre-processing step whenever you're working with the tensorflow data pipeline and then i've gone and built up an entire data pipeline here so we've gone and reloaded our data set and then done the mixture shuffle so we've gone and mapped it we've gone encased it shuffled batched and pre-fetched it which gives us our data set that looks like this 128 samples by 28 pixels by 28 pixels by one cool that is step two now done so we've now gone and visualized and built our data set let's jump back on over to our client time to build the model my guy oh i was born ready to model my guy wait what what are you talking about anyway now what we're going to do is build two models the generator model will be built to try to generate images of clothing and fashion the discriminator will try to learn to spot the fakes so the generator is almost like an artist trying to forge things and the discriminator is an art critic trying to spot them out exactly let's do it radio so we're now up to the good bit actually starting to build our deep neural network now as i mentioned in the client conversation so there's a couple of key components that we actually need to build here so namely the generator and think about this as the part of the gan that is actually doing the generating so we can pass through a bunch of random numbers and it will try to take that set of random numbers to generate some fashion now there is a type of generative adversarial neural network that actually gives you a little bit more control and this is called a conditional gain so you can actually pass through the number one which might map to a specific type of image and it will actually try to generate that image so we're not going to use a conditional again for this tutorial but if you want a video on that let me know in the comments below and maybe we'll uh we'll try to wrap something up that does that so the first thing that we actually need to do is actually import our modeling component so we need to bring in some more dependencies namely we need to bring in the sequential api and we also need to bring in the layers that we're actually going to use so let's go ahead and do that the keras.layers import what do we need okay those are our modeling components that we're gonna need so the first thing that we've actually gone done is brought in the sequential api i think i went through a detailed explanation of how that works in one of the previous videos it was either the image or the eye detection iris tracking video or the one or the face detection one but just know the sequential api means that we take in one input and we get to one output it's basically flowing one way so that is the api that we're going to be using so bring in the sequential api for the generator and discriminator and the cool thing is because the generator is just going to take in a random number of random values so effectively uh a latent set of latent values we can do or use the sequential api for that likewise the discriminator is going to take in an image and try to output a set of binary classification values so 0 or 1 1 or 0 to determine whether or not it is a real or fake image so the generator is going to take in random values try to generate an image the discriminator is going to take in the generated image and maybe some real images and try to determine whether or not it's fake or not and it's sort of like this balancing act between them that's how we actually go about training these bad boys so that's the sequential api then we're bringing in a bunch of layers the layers for the neural network how cool is this that we can actually live in a day and age where we can build ai stuff all right so for our neural network we've brought in from tensorflow.keras.layers import conf2d and then we are also bringing so convolute that's going to allow us perform convolutions i believe we're going to use that in both the discriminator and the generator we're then also going to bring in our dense layers which are fully connected layers a flattened which allows us to flatten a set of dimensions or vectors or matrices we are then bringing the reshape layer which allows us to transform what the output from a previous layer looks like to a different shape pretty useful whenever you're taking inputs of a certain shape so we're going to use this to take in our random variables for our generator and reshape it into something which gives us a little bit of spatial quality you'll see that in a sec we are then bringing in leaky relu so that is just an activation um so leaky relu versus relu so i find it useful to actually visualize what the activation functions look like so a relu is just the this value here so it'll just be the blue line and then following the x-axis the leaky relu has a little bit of a leak on the negative set of values right so that allows us a little bit more data to transfer uh then we've got dropouts a form of regularization and we've also got up sampling 2d which is going to be used for our generator to up sample the images and bring in add in a little bit more depth to or a little bit more space to our image okay so these are all the layers that we're going to need so we'll bring in the sequential api and then a bunch of different types of layers so that is well those are the modeling components that we're going to need and the next thing that we want to do is actually build up our generator so let's actually go and do this so we are going to create a new function and we'll call it build generator and then we're going to instantiate our model so model equals sequential and then we're going to start adding a bunch of layers and what we're going to return at the end is the model right so what we actually need to do in the interim now is actually work out how to build this generator so the first thing that we're going to do is define what number of inputs we're actually going to take in so we might take in let's say 128 values and use that as our random sampling value to actually work out what to generate so that's exactly what we're going to do so let's build our first block and then we'll take a look at what that looks like first up okay that is the first block that we're going to add to our deep neural network so we've gone and added in a dense layer leaky relu layer and a reshape layer so let's see if we can just generate this to begin with so a model let's call it test model for now test model equals build generator and we've got an issue takes two positional arguments but uh this should be wrapped again here one more set of arrays there okay cool all right so that is our test model now defined can we type in dot summary beautiful okay so that is what our initial model looks like and i find it building up this way actually get makes a little bit more sense because you can actually see what's happening so the first thing that we're doing is we're specifying what our input layer is going to be and our input layer is going to be a fully connected layer or dense fully connected layer now the number of units is going to be 7 by 7 by 128 now let me explain why we've gone and specified that we are going to be passing through a 128 random values to our generator to help it determine what to generate from there now this is why i sort of mentioned that you don't have as much control with this type of gain as you do with a conditional gain because the conditional gain is going to focus on what type of image you want to generate with this type again it's really just generating random images from your sample population so it'll be trying to generate random fashion images now the nice thing is that we can avoid something called mode collapse which still ensures that it generates a bunch of random different types of images so it might generate some jumpers might generate some shoes might generate some singlets so on and so forth but just know that that input value that we're going to be passing in which is 128 is what gives our generator some context to determine what to actually generate so we are going to generate based on 128 random values normally referred to as the latent space as well so that 128 is then going to be converted into a spatial area so the 128 values think of it just like an array of 128 values but because we want to eventually convert this into an image we want to give it some spatial qualities right so we're actually going to convert it into a shape of 7 by 7 by 128 and that is what our dense layer is going to do so the number of units that we're going to have is going to be 7 by 7 by our random variable so this could just be our random variable here so model dot add and then we're passing through dense and then seven by seven by 128 we could just pass through the actual value there and then we're specifying what our input value is going to look like which is going to be 128 values then we are applying a activation so we're specifying that we want to apply a leaky relu after the fact this allows us to cater for non-linearities in our model so model.add and then we're passing through leaky value and specifying what that output parameter is so 0.2 so that is what defines how far that bottom bit looks like and then we're reshaping it and this is what now gives us our spatial quality so we're passing through model.add and then we're passing through a reshape layer which takes our dense output which is really just going to be 6272 values and we're going to effectively be converting it to the beginnings of a image so except it's an image which is seven by seven by 128 channels so it's a monster image with a ton of layers eventually we're going to get to the point where we are generating or this is outputting an image which is the same shape as our data it'll be 28 by 28 by one so we're starting out with seven by seven by 128 we're eventually going to get to 28 by 28 by one by adding in a bunch of additional layers so this is our first block so it takes in random values and reshapes it to 7 by 7 by 128. so and just think of this as the beginnings of a generated let's put this on another line beginnings of a generated image julio all right so that's the first set of blocks now done now this is obviously not our model complete because right now it's still outputting seven by seven by 128 which is not in the correct dimensions for a generated image it needs to be 28 by 28 by one so everything that we do from here on out is really to get it towards that shape so in order to do that we're going to be using the conf 2d layers and we're also going to be using the up sampling 2d layers to get us to 28 by 28 by 1. so let's add in a bunch more layers okay that is our first up sampling block so again the up sampling block is effectively going to double this spatial quality so we should be going to 14 by 14 by 128 we're then passing it through a convolutional layer which should effectively condense it down a little bit i believe oh well it's padding same it might not let's actually take a look at that in a second but uh so our convolutional neural network layer is going to be have 128 units that's going to preserve the number of channels our kernel size is going to be five by five and our padding is going to be the same so uh it should effectively not crop but we will see actually it will crop because we're using a bigger filter but we will see in a second and then we are passing through our activation which is going to be a leaky relu again so let's run this so now okay so it didn't crop so now our output is now 14 by 14 by 128. if we really wanted to we could actually just apply the exact same up sampling block so we could literally just take this again and change the number of filters here let me actually show you rather than talking about it right so if i grab this pasted that there and said rather than having 128 kernels here give me one boom boom boom take a look that is our output layer now outputting the same shape as an image so we could actually stop there but there's not really enough typically you want to add in a bunch more layers for your convolute for your generator because this gives it a little bit more sophistication in terms of actually being able to generate something which is sufficiently complex to look like a generated image so we are not going to do that just yet we're going to add in a bunch more layers in between there to give us a little bit more information okay so what we've gone and done is we've gone and added an upsampling 2d block which effectively doubles the size of our output from our previous layer so it's literally going to go 14 let's actually show you let me show you so if i actually deactivated those two additional layers boom boom boom so it's literally going to take that layer and it's going to double it by two so it's performing up sampling so or not doubling but it's actually expanding the values that it's got and i believe it's just duplicating it by it's taking the value next to it and sort of copying it over um but then we apply it pass it through a convolutional 2d neural network to then condense that information and provide a little bit of parametric values or parametric transformations to it because you can see that there are no parameters for our upsampling layer right so let's add our convolutional layer so boom boom boom now we've actually got some parameters that our neural network can learn to be able to generate a better set of values for our output and then we're passing through a leaky relu layer so this gives us activations boom so that's our leaky reality so we're now gone up to 1 million or 1.2 million different parameters for our deep neural network and we're getting closer to what our output image needs to be which is 28 by 28 by 1. okay let's add in another upsampling block and we'll go from there we'll see where up to okay so i've gone and added three more blocks there so let's just say that's down sampling block one even though really isn't down sampling so i've literally gone and copied this up sampling block down here so it's again just copying the exact same like layers which will give us the output of 28 by 28 by what is 128 we're then applying another set of convolutions and a leaky relu and again we're applying the same padding so it really shouldn't just change the size of our output again we're going doing the exact same so this is really just giving us more knowledge or the ability to pass you have more parameters in our deep neural network to learn a little bit better so really it should be 28 by 28 by 128 in terms of our output so if we go and run these boom boom boom right so we're now at 28 by 28 by 128 which means that all we really need to do is add a final convolutional neural network layout convlayer to get to one channel so let's add that final layer and that should be a neural network or at least our generator built cool so that's our final layer so this is going to remember we want it to be one channel so previously we would have had 128 channels based on all the kernels that we had in our convolutional neural network layers because we now have one filter we're only going to get one channel out in our image or our generated image but remember we preserved the shape so we had 28 by 28 and this was really just by passing it through two different up sampling layers we went from 7 to 14 14 to 28 which now gives us so if we go and run this once more we are now at 28 by 28 by 1 with a model which has 2.15 or 2.16 million parameters so we're now looking good so that is our generator now built so we've gone and instantiated the sequential api we then went and did a little bit of reshaping from our input dimensions of 128 random pixels went to 7 by 7 by 128 passed it through an unsampling block which went to 14 by 14 by 128 passed it through another upsampling block which took it to 28 by 28 by 128 and then we went and passed it through some additional it's not really down sampling let's just call this a convolutional block i don't like telling you it's a down sampling block and it really isn't because down sampling would mean we reduce that spatial quality possibly bump out the number of channels we're not doing that right so then we're taking it through two up sampling blocks passing it through two convolutional blocks and we're maintaining the spatial shape because we're setting padding equal the same so two of those which means that we're really just gonna have the exact same values that it's not exact same values but the exact same output shape would be 28 by 28 by 128 and then we are passing it through to our final layer which is going to reduce the number of channels so model.add we're passing through a convolutional 2d neural network it's best find that we only want one filter or one kernel uh the kernel size is going to be four by four we're specifying padding equals the same because that means we're going to preserve that 28 by 28 and we've got an activation of sigmoid because we actually want our values to be between 0 and 1. this is what's going to allow us to generate an image so if we go and run through this now take a look our neural network is 28 by 28 by 1. so that is a generator now built now we can actually go and test out our generator so let's minimize this a little bit so we can actually go and pass through some random values to our generator and we should be able to output some stuff so let's try it out awesome so that is our generator now generating values and effectively images so i went and renamed the model to be equal to generator so that's going to be the name of that model over here so generator equals build underscore generator we can take a look at our summary as we did before and then to generate an image we can pass through generator.predict so use the predict function and then pass through those random values so here we're actually generating four different random images so i've used numpy so numpy.random.rand so this generates a set of numbers using a normal distribution so we're passing through how many images we want to generate so four and then we're passing through those 128 random values which is what our model expects to be able to generate some images and then i've just printed out the array which is over there now rather than visualize or just looking at the numbers we can use matplotlib again to actually view what our random images look like at the moment so we can actually take the visualization script that we had from over here so we can copy that and we can actually just tweak it a little bit to be able to plot this out so we can take so this bit is going to be the same we are we're actually looping through each of our images so let's say um for uh idx comma image in enumerate image and then let's just double check what our image shape looks like so we are 28 by 28 by one okay so that should work so this will grab one image one index we can get rid of the sampling from our data iterator we are just going to be passing through our image here image index should be preserved that's fine we can pass through our index here let's see if that works there we go so those are the images that we've just gone and generated so all i've done is i've taken the exact same visualization script that we wrote right up here and i've gone and adjusted it so that we can now visualize the images that we're outputting from this from our generator model really so what we're doing is we're looping through each one of our images so for index comma image and enumerate our images so this is going to give us an index and the actual value from that specific array we're then using the specific axes from our plot.subplots function and we're using numpy squeeze to squeeze down our values because remember it's 28 by 28x1 by using numpy squeeze we're going to get it by 28 by 28 so it just makes a little bit easier to visualize and then we are plotting out each one of those images so we can see 0 1 2 3. so that's what they look like at the moment pretty crap but once we actually train these this will actually be able to generate a better set of visualizations and better set of fashion components so if we go and run this again we should be able we can get rid of this image because that's causing a nightmare we can delete the shape boom so that's our next set of images so bang bang that's our next set next set right so i can actually grab this put it into here you can say generate new fashion so right now it's looking pretty crap but that's because we haven't changed it right so new images new images new images and you can see it's generating different types of stuff because we are random sampling or producing a random sample to pass through to our initial neural network so pretty cool at least to begin with so we are definitely going to make this a whole heap better let's enable scrolling for that this is obviously going to get a whole heap better eventually once we've actually gone and trained it but that is our generator now done which is one of the hardest parts of actually building a gan model so we've gone and done a ton of stuff so first up we've gone and imported our different modeling components we've gone and built our generator remember we started off with a unique layer which takes in some random values and reshapes it to give it some spatial qualities remember our reshape layer did that we then went and used up sampling to expand the size of our image so we went we actually doubled the size went and applied some convolutional neural network layers to give it a bit of give it some trainable parameters and this allows us to give a little bit more information to what that output actually looks like along with the leaky rallies so what we've then gone and done is pretty much used those same building blocks so here here to be able to increase the size from seven by seven by 128 to 28 by 28 by 128 we've then gone and added a couple of additional convolutional neural network layer blocks and then last but not least this last layer really really important reduces it to one channel which is the same size as our input data we've then also specified an activation of sigmoid to give us a value between zero and one okay that is our neural network looking pretty good the next thing that we need to do is build our discriminator funnily enough this is probably the easier part of building and working with gans because the discriminator is really just a image classifier that is determining whether or not the image is real or fake so if you've watched my image classification tutorial it's really no different to that we're just by adding in a bunch of dropout layers to increase the regularization because image classification is a pretty easy task for neural networks generating new stuff is a little bit trickier so we want to make it a little bit harder for a discriminator to learn so what we're going to do is create a function def build discriminator and we are going to effectively do a similar thing to what we did for our generators we're going to instantiate our model using the sequential api and eventually we're going to return our model and then just to make it a little bit easier to see what we're actually building let's let's set up a similar thing to what we did for our generator so we're going to create an instance of our discriminator so discrim inator equals build discriminator and eventually once we've gone and constructed it we'd actually be able to type in a discriminator dot summary right but right now these two lines aren't going to work because if we are going to run that first up it's not defined oh it actually will work so this line won't work because we haven't actually gone and passed through any layers to that okay so the first thing that we actually want to go ahead and do is build up our discriminator so let's go and do it okay that is the first convolutional neural network layer block so let's take a look so we've gone and ridden three additional lines there so we've added a convolutional neural network layer a leaky relu activation and a dropout layer so the convolutional neural network layer is going to have 32 filters so model.add conf 2d passing through 32 filters the filters are going to have the shape of 5x5 we're not specifying padding equals the same so this means it's going to start condensing down the information that it's getting specifying our input shape is the same as the output shape of our generator so our generator is going to generate an image of 28 by 28 by 1 we then pass that to our discriminator to determine whether or not it's a real or fake image with the end goal for our generator to generate images which our discriminator thinks are real so it's a bit of a trade-off which we'll sort of tweak in our training loop to get to work so the input shape is going to be the size or the output shape of our generator so 28 by 28 by one adding through a activation layer which is going to be a leaky reality leaky rally is actually recommended practice when building uh gans or at least in a lot of them that i've seen they're using leaky releases activations rather than just a straight relu or sigmoid or tan h okay so model. add leaky relu specifying 0.2 as the input parameter and then we're passing through dropout because we want it to make we want to make it a little bit harder for our discriminator to learn so dropout effectively applies some regularization so model dot add dropout and we've specified the positional argument for that as 0.4 so that's the strength of the dropout so that's our first convolutional block and if we take a look once we run through those two new lines so to build the discriminator we can use the build discriminator function and then discriminator.summary allows us to see what it looks like so right now our shape is 24 by 24 by 32 and that is what we're outputting and we've got 832 prime so a very small model at the moment but we're going to build on this so let's add in a bunch of additional convolutional neural network layer blocks and then get this down to what is effectively an image classifier okay so i'm adding a second convolutional neural network layer block and it is pretty much identical to the first one the only difference is i've dropped the input shape because we're not we don't need to define the input shape anymore because it's not the first layer or first hidden layer and i've bumped up the number of filters to 64. i'm probably going to copy this again and create another one with 128 and then let's take a look at what that shape looks like this is going to be our third con block so we're now down to 16 by 16 by 128 so we're getting closer to just having a single output value which is a sigma which has what would it be b sigmoid so zero to one okay so we're getting there so let's go and i'm going to add in a bunch more layers but basically it's going to follow this similar pattern we're then going to flatten it down and then output a single value using a dense layer so let's do it okay so that is my fourth convolutional block i don't think i'm going to add in another one i think we're good for that i'm just trying to mimic the exact same model that i pre-trained earlier so you guys can use the trained weights that i've got so that is so effective we got four different convolution blocks so the first one with 32 filters second one 64. third one with 128 last one with 256 now what we're going to do is we're going to flatten this model actually let's take a look at our apple first so if i go and run this run this run this we have an output of 12 by 12 by 256 so by flattening it we'll actually get 12 by 12 by 256 being passed to a dense layer let's do that so flatten then pass to dense layer so model dot add and then we're going to pass through a flattened layer it should flatten it down we are then going to pass through a dropout layer so model.add drop out 0.4 i wonder if it's best practice to add a drop out after a flat anyway we're gonna i've added it in the pre-trained model so we kind of need it now to add might take a look at that let me know if you get better performance by removing that layer but for now we're going to leave it so we are now going to output a single value and our activation is going to be a sigmoid because we want the output value to be between zero or one so one i can't remember the way i configured it whether or not one is representing a false let me just double check so yeah so one represents a false image or one is eventually going to represent a false image 0 is going to represent a true image so again you can play around with that anyway activation equals sigmoid and that looks good i think that's our model done yeah cool all right so that so you can see that we're originally taking a image which is of the shape 28 by 28 by one and then we're finally going all the way down to the bottom and we're outputting a 1 which will be a value between 0 and 1 to represent true or false whether or not the image is fake or real one being a fake image zero being a real image so and again different implementations run these different ways one might be a real image one might be a fake image it very much depends on how the model is actually set up now we could actually take those images that we actually generated from our generator right this is what's cool about it you can prototype some of this you can take this image over here and pass it through to our discriminator and it should output a zero one so if i went discriminator let me bring this up this grim in nato i can never type this dot predict and pass through image uh we've got issues all right so uh negative dimension size chords by subtracting one what have we gone and done there do we need to pass through dot predict no we've got an issue all right let me debug this wait we've got one image inside of image.shape so the model is okay wait i thought we had batched this so for image dot shape okay that should work so i'm guessing that we're getting an error because we're passing through a single so it should be four images now okay there we go that might works much better so don't predict okay makes more sense so the reason that we're getting that error is because effectively we're trying to run a single image through to that neural network which is going to throw an error so it's effectively like doing that so you can see that that's our image if we take a look at the shape it's 28 by 28 by run if i run that typically when you're passing through an image or a single image to a deep neural network you it expects a batch number to be first so because this is a single value it's a single image in a batch so we'd actually need to type in np dot expand dims pass through the parameter zero fixed so now we can generate a prediction on a single image which you can see 0.5 so it's not very confident whether or not this is real or not but again we're going to train a better generator once we get through that now if you've got more than one image you don't need to go and wrap it in expand dims you can actually just go and do this so if i go and generate a new set of images from up there and then run it through our model run it through the dot predict function should get no errors boom take a look at that pretty cool right so that is our generator and our discriminator now built and you sort of saw how i'm building up these layers to ensure that our input and output are of the correct shape so remember our generator is going to take a set of random values which in our case are going to be 128 random values and it's going to output an image or a vector or a matrix of the shape 28 by 28 by 1. now our discriminator is going to sort of do the opposite of that it's going to take the output of the generator so 28 by 28 by one and it's going to output a single value between zero and one to determine whether or not it's real or fake cool okay that is our set of generators now done let's go jump back on over generators and discriminators are done let's go and have a chat to our client so that's done what about the custom training loop ah that's one of the most important parts when building gans we'll write a custom training loop to train both the generator and the discriminator simultaneously ah got it let's go so we're on the home right now but this bit is pretty hardcore i wanna disclose that but we're gonna take it step by step so the training of gans is notoriously difficult and nobody actually really tells you why it's difficult but i'm going to give you my feedback so the reason why it's difficult is you need to find a balance between the speed at which the discriminator trains and the speed at which the generator is able to learn so a big thing that we need to do whenever we're training these side by side is ensure that our discriminator doesn't train too fast so that it's able to smash out and know what a fake image is every single time so balancing that is uh really really difficult and is really really important to ensure that you get that right one way that we help do that is we actually inject some random noise into the outputs from the discriminator almost to kind of like trick it and slow it down a little bit now the other thing is that it takes quite a fair bit of time to train one of these that is actually generating good output so um if anybody goes hey you can train one of these in like 10 minutes then be very very skeptical that's just my feedback and again you might get different output from some experts out there in the field um but that that's just my experience so just keep those two things in mind but we're going to take this step by step and go through it so consult constructing a training loop now what i mean by training loop is so typically when you go and train a deep neural network using tensorflow or pytorch you can typically there's a couple of key steps right so you instantiate the model so maybe create uh modeled equal sequential and then model. add a bunch of layers then model.compile which actually assigns a loss function and an optimizer and then model.fit which is actually what goes and trains the model now because discriminators because scans have two components or a generator and a discriminator you actually need to train them side by side which means we can't just use dot fit we need to do something a little bit more tricky to ensure that we get this to work so what we're going to do is define our own custom training loop and by training loop we're just changing what the dot fit function does by defining our own training step so we're going to do that over here and we're going to do it using a subclass model you can also do it using at tier function decorator there's some good documentation on the tensorflow website on that i'll link to it below but for now we're going to do it this way and as per usual all this code is going to be available by github so don't stress if you don't get it you can go back through and take it at your own pace okay first thing we need to do is set up some losses and optimizers so um in order to do that we actually need to bring in some losses and optimizers so let's do that we're going to use the uh what i think we're going to use what is it binary cross entropy think we're actually going to use binary cross entropy for both because yeah now we'll come back to that we're going to use binary cross entropy and we're going to use the atom optimizer so first thing let's uh let's import those so i'm going to import some optimizers from keras so from tensorflow.optimizers uh we are going to import adam and then from tensorflow.keras dot losses we are going to import binary cross entropy cool so our atom optimizers is going to be the optimizer or both and binary cross entropy is going to be the loss for both now you're probably thinking nick what the hell how are you going to use binary cross entropy when your generator is outputting an image well what we actually do with that image is we pass it through to our discriminator and we determine whether or not a generator has actually produced an image which is able to trick the discriminator so we actually reward it for getting our discriminator to produce the wrong value which is kind of not intuitive but once you actually play with these a bunch of times you'll actually be like oh that's actually kind of cool so we're actually rewarding our just generator for tricking our discriminator and we're rewarding our discriminator for getting it the right one well determining what type of image it actually is cool all right so those are our losses and our optimizers for from tens photo carousel optimizes import atom you can try stochastic gradient descent or another type of optimizer atom is probably what i use most of the time and then for our losses so from tensorflow.losses import binary cross entropy pretty cool right like so if you actually go and build some other i know i'm veering off into the wayside but when i was actually building the super resolution model i actually had to use a combination of different types of loss functions so i actually use mean squared error for the perceptual loss and that is how different the actual values are from a real super resolution image versus and not a super resolution image and what else did we use um perceptual loss i think i used no i didn't use binary cross entropy but i used a combination of a bunch of different types of losses so we actually took the discriminator loss versus plus the perceptual loss as the the final loss function but again i'll probably go through that once we actually go through that model okay so that is our we've imported our optimizers in our losses now we need to create instances of them so we're just going to create a sample of examples so we'll go g opt equals atom and we're going to set the learning rate equal to 0.0001 so our learning rate for our generator is going to be faster than the learning rate for a discriminator because we don't want our discriminator to go too fast and absolutely smash our generator and this is part of the learning process right so you might need to tweak this if you're doing different use cases but i found that this has worked all right so we're defining our two optimizers so i've typed in g opt equals atom and i've specified a learning rate of 0.0001 and then for our discriminator i've created a again an optimizer so d opt equals atom and then our learning rate is zero dot zero z so there's just one extra zero so zero zero zero zero one cool and then i'm gonna define my losses so g loss equals binary cross entropy and then d loss also equals binary cross entropy boom okay those are our losses and our optimize is now set so again the reason that we're specifying the learning rate for the discriminator slower is just so it doesn't blow out and learn way too fast and cause our generator to lose the plot and we've also gotten to find our losses so that is step 4.1 now done so let's set up our losses and optimizes now as a big boy so setting up the subclass model so first thing that we need to do is import the model class from keras so from tensorflow tensorflow can't type tensorflow.keras dot models import model and this is the base model class so once you get to this level you you've got a ton of control over what tensorflow does so you can do a ton of stuff if you want to so this is importing the base model class to subclass our training step cool all right so i've ridden from tensorflow carousel models import model now when you're defining new models there's basically like three two keys functions so we're gonna create again so class fashion again you can name it whatever you want and we are going to pass through our base class and then there's a couple of key methods so again method is inside of class so there's a couple of key methods that you need to define when you're subclassing a model so def init because we need an initialization function we'll come back to that def train step and this is what actually does our training you can also define and so keep in mind train step is called when you call the dot fit function right so this is really really important to note if you wanted to run an evaluation function you'd also write create a def test step method we're not going to use that but just a key thing to note we also need to have a compile method so def compile and again we'll come back to that so those are the key methods that we actually need to define in our subclass model so and the reason that we're going to define a subclass model is because we're going to do some fancy crap inside of this test step model or test step method the other way that you'll see people do it is they'll actually write um at oh gosh at tf function and then they'll actually write a custom test step or custom train step they'll do it sort of like that and then they'll do a whole bunch of other stuff but um for now i'd just like a little bit more control with the subclass model but just keep in mind that you've got a few different ways to actually go about doing this okay so first thing that we need to do is go and pass through some parameters to our init method so let's actually go ahead and do this alrighty those are our parameters that we're going to be passing through to our init method so we're passing through our generator passing through our discriminator we're passing through any positional arguments in any keyboard argument so this gives us a little bit of flexibility if we want to go and inherit from our base model then we're going to run super dot init and we're going to pass through any of those arguments so again if we want to go and use some of the base functionality inside of our keras model we've got the ability to do that now so we've passed through those arguments and keyword arguments to our base model to our base class effectively then the next thing that we're going to go ahead and do is set up two attributes for our generator and our discriminator so that means that we're going to be able to refer to them inside of the model so let's actually um so create attributes for gen and discriminator self dot we'll call it generator equals generator self does disgrim in nato equals discriminator uh we don't we can get rid of the pass i think that's it for our init method yeah that's pretty much it so we've gone and ridden what have we gone and done so we've gone and passed through our generator and discriminator model so we'll instantiate those before we actually go or we'll instantiate those then pass them to our subclass model and then we're passing through any keyword arguments and any positional arguments to our base class so this um plus through args and kw args to base class and then we're creating attributes for our generator and discriminator so that is our init method now done now the next thing that we need to do is actually go and compile so to this we're going to need to pass through our optimizers and our losses and we're going to need to run super.compile so let's do that cool all right so that is our those are the input arguments that we're going to be passing through to our compile method so we've got def compile and then we're passing through our generator optimizer a discriminator optimizer a generator loss and our discriminator loss we're also going to be passing through any arguments and keyword arguments we might need because we're going to call so compile with base class so we're going to run super dot compile and we are going to pass through our arguments and our keyword arguments boom okay and then we're going to create attributes for each one of these so attribute for our optimizers and attributes for our losses so let's go and do that okay that is our compile method now done so let's quickly recap so we've gone in to find our knit method and our knit method takes in two key things right takes in our generator and our discriminator models that have already been instantiated you can ignore the super bits for now just know that they're necessary evils right so we're going to be passing through any additional arguments that we want to pass through when we create this model via that so then we're creating attributes for our generator and our discriminators we're going to be able to refer to them as self.generator self.discriminator likewise we're doing something similar for our compile method so we're passing through our optimizers so g op d op g lost d loss and remember we've already created those over here right so we're effectively passing through atom atom binary cross entropy binary cross entropy and then we're creating attributes for those inside of our fashion gain class and again this could be a different you could name it whatever you wanted to it's just the naming convention that i've gone and taken or the fashion game okay so those are the two easier bits now done so we've gone down the init method and the compile method i know it's always a little bit painful so all right now we're going to get onto the big boy so for our train step the first thing that we need to take in is our batch and this is a batch of data keep in mind a batch of data in our case is going to be 128 images of the size 28 by 28 by 1 and that has been defined right up over here so when we went and created our data set up there in step or section two now what we need to do is actually do something with that data and that is exactly what we're going to do inside of our train step so the first thing that we're going to do is we're going to get a batch of data so we'll call it real images equals batch so this first step is get the data right so we're getting first things first we're getting a batch of real images and then we're going to get a backup batch of fake images or generated images right and this is going to be self.generator tf.random.rand i will just call it dot normal and this is effectively what we did previously right so we went and created some random images remember we used mp.rand.randn let me show you how did we do it it was no that's the data iterator we did it over here so we called mp.random.random pretty much just the equivalent of that with using the tensorflow library is tf.rand.normal tf.rand is it random random.normal and we'd be passing through i don't know like 6 comma 28 by 28 by 1. i can do it probably needs to be wrapped there you go so what we're doing is we're effectively generating those random input values that are going to be going into our generator right so again these actually wait it's not 120 it should be um six by 128 by 1. those are random values so we're passing through random values and we're sending those through to our generator model to be able to generate random values or generate random images to begin with and this is because we need real images and we need fake images in order to go on ahead and train our model so let's go and do that so we are going to generate let's generate 128 so we've got the same number of images in our real images as our batch so 128 by 128 by one and we can close that and then we're going to specify training equals false because our deep generator is not training at the moment we're just making predictions okay so that is that now the first thing that we're going to do is train our discriminator so train the discriminator so there's three key steps that we need to do in order to go and train our discriminator so the first one is we are going to pass the real and fake images to the discriminator model we then need to create labels for real and fake images we are then going to pass through or add some noise to the outputs and this is what helps it from learning way too fast we then calculate the loss and then apply back prop propagation and this is what effectively allows our neural network to learn right and i'm just going to tab these in okay now in order to actually go and train our discriminator we need to start calculating our gradient so we can do this using the tf.gradient tape method so let's go ahead and do this so with so that is setting up our gradient so this is going to start calculating uh or monitoring each one of our functions the next thing that we want to do is actually this next step so pass the real and fake images to our discriminator model so let's do that okay so those are the initial predictions right so we've gone and written three lines of code there so let's say break that down so the first line is actually taking our real images and we're just passing that through to our discriminator and we're storing the values inside of a variable called y hat real so y hat real equals self.discriminator we're taking these images from over here and we're passing it to the discriminator model that's all we're really doing we're passing through training equals true because this means that the dropout is going to activate right so if you pass through training equals false dropout layers aren't going to run then we're also doing the same for our fake images so y hat fake equals self.discriminator we're passing through our fake images from over here and again we're specifying training equals true and then we're combining it into one set of outputs so y hat real fake equals tf.concat so this just concatenates them both together and we're going to be passing through our y hat real and y hat fake so that and that and we're combining on the zero axis so it's going to combine all the samples together cool so that is the first step now done so we've actually gone and produced predictions as to whether or not our discriminator thinks the images are real or fake the next thing that we need to do is actually create labels because remember this is effectively supervised learning problem at the moment we're determining whether or not they're real or fake zero or one so let's go and do this okay so those are now our predictions so we've actually gone and assigned the labels and really you could go and pre-gen well actually can't really pre-generate this because we're generating new predictions each and every time but that's fine so we've now gone and created a new variable equal y underscore real fake so this is the equivalent of this over here right but these are the predictions from a discriminator these are the actual labels so why underscore real fake and i'm just setting the order so you know what order they're in so in this particular case again we're creating a set of labels for our real images so and the real images are going to be zero so really our discriminator is all about spotting fakes and we've set that equal to tf dot zero's like and we pass through y hat reel the cool thing about this is it effectively goes and generates a set of zeros based on the shape of your input so in this particular case we're passing through our y hat image or y hat real values and we're saying all of these should actually be zero so it's going to be in the same shape as our real or our y hat predictions but they're going to be zeros now we're going to do the same thing for our y hat fake so tf dot ones like we're passing through y hat fake so effectively we're gonna have 128 zeros and 128 ones over here and those are going to be the labels for our or the real labels for our predictions from our discriminator so think think of this as your true y values and we're going to concatenate both of those together using tf.concat so our final value is going to be y underscore real fake okay so we've now gone and made our predictions we've now gone and created our labels and let me just quickly show you what xero's like actually does so if i actually go and wrap this set of actually let's not do it with the that's our latent variables so let's just enable that um so what would it be it would be um it'd just be like something like that right so to be predictions like that so if you can see that we've got a bunch of random values right so if i type in tf.one's like and pass that you can see it's generating a bunch of ones but in this same shape over here now if i did tf 0s like it's generating a bunch of zeros now if i go and concatenate those so tf.comcat that and if i put this inside of an array and let's say one's like and let's close that array and then type in dot access equals zero so you can see we've just got a massive array and we've got a bunch of zeros and a bunch of ones so that is effectively what we've gone and done in this line now what we need to do is add some noise to these outputs and then we can calculate loss and apply back prop and that's the discriminator training component now done cool so that's our noise added to our white white true value so in this particular case we've gone and added a random value between and we've multiplied it by a weighting of 0.15 for our real images and negative 0.15 to our fake images so i played around with this a bunch when i was actually testing this out and i found that adding values to our zero values and reducing values from our one values seemed to work a little bit better otherwise it threw our loss metrics out so um i've you typically see people just adding a random value to both not necessarily adjust based whether or not it's a zero value or one value so true or false so in this particular case i've gone and customized this a little bit but let's go through these lines so there's three lines here so noise underscore real equals 0.15 multiplied by tf dot random not uniform so it's a uniform distribution and we are adding the or creating the values in the shape of y hat real so we're going to match that and then we're adding noise for our fake values so noise underscore fake equals minus 0.15 multiplied by tf.random.uniform and then we're passing through the same shape as our tf as our y hat fake values from over here then we're adding it to this value so we're adding it to our y true values so why underscore real fake plus equals tf.concat so we're effectively just concatenating both of these values back so plus equals tf.concat and then passing through noise real noise fake so that is injecting those noise into the true outputs cool all right so that is that now done now all that's left to do is actually calculate our loss and apply a back prop so let's calculate our loss so total d underscore loss equals self dot d plus which is going to be binary cross entropy remember binary cross and then we want to be passing through our real that's why real fake not y hat y underscore real fake and then y hat real fake cool alrighty that is our loss now calculated so why real fake y hat real fake and we are going to calculate that then what we want to do is we want to apply back prop so we're going to go outside of our tf.gradient tape loop and we're going to first up calculate our gradients and then we're going to apply our gradients using our optimizer so let's do this and then that's our discriminator trend cool that is the training step right discriminated now done so i know we wrote a ton of stuff there but again this is pretty hardcore in terms of deep learning so the first thing that we did is we went and calculated our gradients so we specified a new variable called d grad and we set that equal to d dot or d underscore tape dot gradient so remember d are using the tf.gradient tape function allows us to calculate all of the operations that are happening by using our deep neural network and that means that we can use that to calculate our gradient with respect to our loss the d underscore tape.gradient and then the first parameter that we pass through is our loss so remember this is our binary cross entropy and then we want to calculate the gradient for each one of our trainable variables so we're going to extract our trainable variables using self.discriminator dot trainable underscore variables and then what we're going to do is we're going to apply our gradients using our optimizer so self.deopt because remember we passed through ad optimizer over here which was an atom optimizer and then we can use dot apply gradients to actually go and apply back prop and so to that we pass through zip because we want to do our gradient with respect to each one of those trainable variables so zip and then we're passing through d grad and then again all of our trainable variables so it's effectively going to look at the gradient for each one of those variables and apply backprop using our learning rate okay that is our discriminator now done so i know that that's the hardest bit just keep that in mind that is the hardest bit of the training step so first thing that we went and did is again we went and passed our real and fake images to our discriminator model and we went and concatenated them together pretty self-explanatory not not too crazy there we then went and created labels for our real and fake images and remember our real images are gonna have a zero label and our fake images are gonna have a one label you can tune this you can do it slightly differently if you wanted to switch them around to keep in mind you've got to change the noise as well and change um i think you can switch them around you'll be you'd be okay you don't need to go and do anything else but just keep in mind for now that's how we've gone and set it up because it's important because of how we're actually going to go and set up our generator then we inject a bunch of noise so the noise is really just uh being applied to our true outputs and this gives us um our discriminator a little bit less certainty about whether or not it's predicting correctly or not so it confuses it a little bit so we then go and pass just a random set of values so we're using tf.random.uniform and we're scaling that by 0.15 for our real outputs because that adds some values to our zero to our zero labels and then we're subtracting 0.15 multiplied by tf.random.uniform to our fake labels we then calculate our loss using self.d underscore loss which is binary cross entropy we then go and apply backprop using the gradient tape from up here and the apply gradients method from our atom optimizer okay that was pretty brutal but again it's quite difficult and that's the hardest bit done trust me that is the hardest bit about actually going and setting up a custom training loop against and it's probably the hardest one that you'd find in a lot of deep neural networks that you'd actually have to go and build so congratulations you just went and wrote that and you're doing well all right now all that we've got to do now is actually go and set up a training step or the the training component for our generator now so it's going to be similar but it's not by no means as complicated as what we just wrote so let's go and do a generator now so again we're going to use our gradient tape so with tf.gradient tape as g tape now let's actually write so let's actually call it uh to train the generator okay so the first thing that we need to do is generate some new images then what we're going to do is we're going to uh apply or create the predicted labels and this is a little bit counter-intuitive but i'll explain it and then again we're going to calculate loss and then over here we're going to apply by backprop cool alrighty so let's go ahead and do it so that is our new set of images now generated so i've just gone and created a new variable called gen underscore images and i've set that equal to self dot generator and again it's pretty much the same as what we're doing up here but this time we're setting our training parameter equal to true so self.generator and then we're passing through tf.random.normal or passing through 128 by 128x1 so this will have 128 random variables which are passed to our generator and then we'll specify training equals true now we're going to create our predicted labels now this is a little bit counter-intuitive so over here when we actually went and created our discriminator what we actually said was that fake values had the label of one now because we're trying to trick a discriminator what we're actually going to do is we're going to set the variables for our generator here to zero so what we want our discriminator to actually lose out on is we want it to think that our generated images are actually real images so that's how we're going to calculate our loss for our generator so you'll see that in a second so let's actually go and do this okay those are the next two lines of code written so i've gotten written two lines of code there so the first thing is we're running our generated images through our discriminator so predicted underscore labels equals self dot discriminator and we're passing through our generated images from here and we're setting training equals false now because we don't want our discriminator to be learning whilst we're actually training our generator so that's just a nuance of how you actually go and set this up so that is going to generate the predicted outputs whether or not a discriminator is determining whether or not our generated images are real or false and remember a discriminator will output a 1 if it thinks the image is false or not so over here you'd expect all of these to be ones but this is where we try to confuse our discriminator because what we're actually saying is that our generator loss is rewarded every time it thinks that our discriminator is or every time it thinks that our generated images are actually real so when we go and calculate our loss total underscore g underscore loss is equal to self.g loss and we're actually saying that the the real labels are actually zeros so we're actually saying that the real image or generated images are actually fake images and this is something that took me a little while to get my head around but this is actually how you go and train the generator so tf.zeros like and then we're passing through our predicted labels and then we're passing through what our discriminator actually predicted so whether or not it actually predicted true or false and then from here what we'll go and do is back prop and then we'll train our generator so that it's able to produce better images every single time and this is that that's the trick right trick to training to fake out the discriminator because over time what our generator will learn to do is produce the images which are better at faking outputs through our discriminator so it will actually start getting zeros by passing it through our discriminator because our discriminator won't know the difference between a real image and a generated image so all that's really left to do is actually go and calculate our gradients apply back prop and then return our loss metrics then we should be able to train so let's go and wrap this up alrighty cool so we're now going and applying backprop this is almost identical to what we did for our discriminator first up we calculate our gradients so g grad equals g underscore tape dot gradient we pass through our losses the first metric we pass through the variables that we want to calculate the gradient for so self.generator.trainablevariables nice spelling there be a trainable b-a-r-i it should be that this is spelt wrong did i spell it wrong over here v-a-r-i-a no that's good all right so then we're passing through what we want to calculate our gradients for and then we're going and applying our gradients using our optimizer self dot g opt dot apply gradients we're zipping them because we want to do them both at the same time so zip grad and then the trainable variables so self.generator.trainablevariables all we then need to do is return and we haven't gone and tested this out we've gone and written a lot of code and not tested all we need to do is then return our loss metrics so d underscore loss is equal to d lot should be total d loss yeah total and this is important when actually monitoring the model because you want to ensure that the loss metrics are reducing steadily to together in terms of they're not one is not blowing out versus the other they're they're sort of reducing or they're actually staying stable you don't really expect them to reduce all that much to be honest um so g-loss is total g-loss perfect i think that is done who knows we're probably gonna need to do a bunch of debugging for that but again all this code's going to be available because we wrote a ton that is our full subclass model so fashion again all of that quite a ton don't ask how long it took me to learn this stuff but i'm trying to teach it on to you all right cool let's uh we've got our first error so in line 178 it's a predicted label so over here we've got an error and i'm guessing it's because i haven't closed this it is and we've got another error so total g loss what's happened there so we can get rid of that so it looks like we just got misplayed all right cool there we go so we've now gone and created our subclass model so it doesn't look like we had any other syntax errors but training errors is a completely different beast so let's actually go and now kick so we actually want to create an instance of this model and then we're going to set up our callback train it and review performance so let's kick this off cool so we've gone and created an instance of our fashion game class so create instance of subclassed model and so what i've written is fashgen equals fashiongan so it's an instance of this big bad boy class over here and then to that what we're doing is we're passing through our generator and discriminator because that is what our init method expects over here then we're going to compile it and pass through all of our losses and optimizes so let's do that and what order are they expected in we i think we can actually just copy these because we've named the variables the same so uh g of d op g lost d loss g of the object cool we can do that cool so that's a compiled successfully so fastgan.compile and then we're going to pass through our generator optimizer a discriminator optimizer at g-loss and our d-loss is this still zoomed in yeah it is okay that is that done so that is step 4.1 now done i know that took a while and there was quite a fair bit we can get rid of that that is our uh our custom training loop and our subclass model now done now with the callback i'm just going to copy this over from because i've used this a bunch of times and this is actually from i think it's francois chorley's sample i actually tweaked it a little bit just to get it to work but i'm going to walk you through it so we are going to import a bunch of dependencies so import os so this just helps with folder navigation and then we're going to import the eraser image function so from tensorflow.carous.preprocessing.image import array to image and then from tensorflow.com callbacks import callback so that allows us to create our own custom callback on epoc end so if i go and import that and then i'm not going to write this but you can use this a ton of times for gans i'm just going to show you what you need to tweak so this model monitor is going to allow us to save examples of our generated images as we're actually training so there's two key parameters that you want to pass through to it the number of images that you want to generate and how big a latent dimension should be so remember the latent dimension is just the random values that you're passing to your generator to generate a random image now on epoc end it's going to go and do a bunch of stuff so it's actually going to generate random values it's going to pass that to our model generator and then it's going to generate a bunch of images interesting that we're using uniform yeah we could actually go and do that right up here but for now i don't want to mess around with it but i wonder who knows maybe we'll do that later on um okay so we're going in generating our images we are then going and converting it to a numpy array and we're saving it now this is the important bit that i wanted to talk about in terms of actually saving your images you need to create a folder called images and this is going to this is where these images are going to save to as part of this callback so inside of your root folder so this is the notebook that i'm currently working on just create another folder called images so it's literally a new folder call it images and that is where your images were saved to if you wanted to save it somewhere else let's say for example you wanted to call it gan images all you need to do is change the path or the folder that you want these to go into in this case i'm just going to save it into images because that's why i've got it configured but you could change that there again all this code is going to be available and this is optional you don't need to go and use this callback i've just found it a little bit easier when actually training our model so i'm going to create that and then we can train so these two are really sort of optional but they definitely help when monitoring your model okay training so we've now gone and created our subclass model created our generator discriminator we've gone and created a callback this is how we actually go and train so here's the equals fashscan dot train oh actually it's not fit not trained god i'm falling asleep uh and then we pass through our data set because remember we created a data set does ds dot as numpy iterator dot next so that is our data set let's take a look at the shape it's 128 by 28 by 28 by one so we pass through our data set and then we specify the number of epochs that we want to train for i recommend 2 000 we're not going to train it for 2000 because it's going to take ages but recommend 2000 epochs which is gonna take a long time i won't doubt it i'm not gonna lie it's gonna take quite a fair bit of time we'll probably train for 20 and then i'm going to give you the pre-trained model that you can use as well if you want to use this callback you don't have to but if you wanted to use it you can pass through callbacks and then pass through model monitor which is the name of this callback over here cool alrighty cross your fingers because we wanted to wrote that entire training loop without testing it so there might be some errors we'll actually see so if i go and run this we've got an error already all right what's happening sequential object has no attribute trainable variables so i'm pretty sure i've spelt that wrong and you can see i definitely have where was this line kicking up it was in d opt yeah i thought i wrote it wrong so trainable var hope it's there yeah so you can see i've got an error there so that's a typo copy this could be that v-a-r-i-a-b-l-e-s okay let's run that again recreate our fashion gain instance recompile got another error and got an unexpected word training self.generator okay we've closed that off incorrectly that's fine um so that is here so the reason that we're getting an error there is because we are effectively passing training equals true to this tf.random not normal so you can see i'm missing a closing brackets there that's better remove that one there okay so let's run that recompile try to fit now it's looking more promising there you go we're now training so what you should notice is that your d loss and your g loss sort of balance they're not go no one is going to blow out and or ideally you don't want one to blow out and you don't want one to decrease really fast and you don't want one to increase really fast you want it to sort of kind of stay steady and this is what you're kind of noticing here right so it's sort of bubbling around 68 67 and likewise we're sort of getting that around for our g-loss as well it might increase occasionally but ideally you want it to be stable over the longer term so what we're going to do is we're going to let that train uh and let's give it a second just to ensure that it generates some images because remember it's going to generate an image on epoc end which is dictated by this callback over here so if i go into our images you can see it was generated seconds ago that is our initial image it's literally just this random black square with little white bits on the corners so nothing crazy at the moment and this is common so it'll suck for a ton of epochs and then it'll get way better so just keep that in mind so you can see where our generator is already spiking not spiking massively but it's definitely increasing so we'll let that go and then what we'll do is we'll review model performance i'll load up the pre-trained model and i'll give you the ability to go and download that yourself and then we'll go from there we'll be back five minutes later all righty that is the beginnings of our gan now train so if we actually take a look at the loss you can see that the discriminator started to stabilize a lot more versus our generators starting to reduce and loss which is a good sign perfectly okay if you do run it for the full 2000 you'll see that they start to converge and they sort of reach this happy medium so um that is perfectly okay as long as you're not getting like one blowing out one going massively low and so ideally you don't want like what you don't want to see is one like let's say for example your discriminators training and then it goes to 0.00001 loss and the generator spikes out of control that is probably an indicator of mode collapse or the fact that that just one of the training rates is way too high or low um and you've got to stabilize and you got to stabilize it out but uh but now it looks like it's okay and again i'm going to give you the the pre-trained models as well the one that i went and trained for 2000 epoch so you can test that out but before we do that let's quickly go and review performance and then we'll jump back on over to our client so because we saved our training inside of a variable called hist we can access our performance so if i type in here history you can see we've got our discriminator loss we've got our generator loss so we can actually go and take a look at those so let's actually create a plot perfect alrighty so there you can see it so what's happening is our discriminator has sort of steadied towards let me zoom in on that so our generator has spiked up and but you can see it's definitely coming back down and this was pretty common i saw this a ton of times keep in mind we've only trained for 28 bucks which is by no means the end state you'd have to train for a ton longer to get a really really good model i think i actually did the full 2000 epochs but you can see that our discriminator sort of bounced around down the bottom at 0.2 something what did it actually have 0.26 would seem to be about right um now a generator is coming down so it means that it a generator is learning how to generate real images which is a good sign so that is our training now done you can obviously train for way longer and you will get much better performance um as long as you don't see one of these spike up and the other spike down okay that is looking good let's jump back on over to our client and then we're going to wrap this up by testing it all out oh actually wait before we do that we've also generated a ton of images right so we can actually take a look and see what this is looking like already so it looks like we've got a bunch of white images so again pretty crap at the moment which is why i'm probably going to show you the final train model all right so you can see we've got some dots so let me zoom in so you can see we're getting images but they're right now they're white which is a really poor indicator of performance these are sucking but again as you train for way longer you're going to start to get significantly better performance because you can see this is uh epoch 19 which is effectively epoch 20 it's starting to learn how to generate images right from the bottom but that is literally how it starts it takes a long time to get one of these fully trained up i think i actually did spend the full 2000 epochs okay let's go back to our client have a chat and then we'll go from there all that's left to do now is test it out sweet let's do it we can use the generator model to make predictions and visualize those tiny images using matplotlib to save you some time i've also pre-trained a model that you can use which will be available on github guys nice alrighty so now that we've gone and trained our model let's actually go and test it out so before loading up the pre-trained model let's actually just test out with our baseline model so we can actually load in weights for pre-trained models and i'm going to give you those weights i'll try to upload them on github so you can test this out first thing that we need to do is actually go and make predictions so we can create a variable called images and use the generator which is now trained through our subclass model so generator dot predict we can use tf.random.normal and then pass through the number of images we want to generate so let's generate like 16 pass through the latent variables pass through one because that is the shape it is expecting and that should generate images images perfect so those are our images now generated so you can see those there then the next thing that we can do is actually visualize them using matplotlib so rather than just looking at a random set of arrays we can actually go and visualize them a little bit easier so let's go and do that perfect and that's what we're getting out of our generators are pretty crap at the moment but again this has been trained for jack all time 20 epochs which is absolutely nothing so let's actually take a look at what it would look like with a trained model so one that's actually been trained for quite some time so rather than using the base generator what we'll do is we'll load up some weight so i've got some weights sitting in a folder called archive so i've got the discriminator weights and the generator weights we don't need the discriminatory it's really just a training thing but let's actually load up generatormodel.h5 so we can go uh generator dot load weights and we are going to go os.path dot join archive and then it should be generator model.h5 perfect so that looks like it's loaded up so if i go and run this line now boom so you can see we are now generating fashion so again exact same architecture the only difference is i've gone and trained it for 2000 ebooks but you can see that by doing or running through this type of architecture you can actually do quite a fair bit obviously it takes a long time to train so what i was typically doing is leaving my computer training overnight or actually just spinning up a collab instance with background execution which i think you might need collab pro for but honestly it's pretty worth it considering that these take quite a fair bit of time to train if you found better results by training in different architectures or better ways of training faster do let me know let's actually make this a little bit smaller so we can see it you can see that we are actually generating images of fashion so we've got pants we've got bags we've got jumpers we've got shoes i don't know what that is i think it might be a bag again we can go and generate more images so again we've got like sort of sneakers now i don't know slightly different type of bag slightly like a men's boot or something there or a sandal let's go and generate more but you can see that this is obviously super powerful so like we've done it on fashion but you could do it on just about anything um obviously keep in mind that training is going to be a big factor in the amount of compute that you're going to need it's going to be a huge factor and you saw what our model performed like when we were training using the old baseline model and not a pre-trained one but i will make these weights available to you via github so you can actually test this out but you've seen what the performance looks like once you've actually gone and trained a lot longer so i think when i was training this i actually managed to get the generator loss to pretty close to what the discriminator was so i think it ended up balancing out at around 0.6 for each one of them so it was bouncing around 0.6 so that's just an indicator of what you might need to get to in order to get something that actually produces images that look like this last thing that we want to do is just in case we want to use this later on if we wanted to you could actually just save the model and we can do that using generator dot save and then we can pass through oh uh we can pass through whatever we want the name to be so we'll call it generator.h5 you can save the discriminator as well this grim inator dot h5 boom so you can see if we go into our base model we've now gone and saved the discriminator and the generator that is this done in a nutshell so again we've gone and done a ton of stuff in terms of doing this so first up we imported and installed our dependencies we went and visualized our images remember these are our real images you saw what our generated images look like below we took a look at data sets building a neural network creating a generator and you saw that effectively your generator is passing the outputs to your discriminator we then went and built our discriminator and then we built this monster training loop which obviously takes quite a fair bit of effort uh we took a look at the what is it the callback and training and again the length of time that we train for is by no means long enough to actually get a great result i'd recommend 2000 epochs we did 20 but time permitting i've only got so much time to generate these tutorials so i figured i'd at least try to get this out for you guys um and show you what a pre-chain model does look like because i went and pre-trained this a ton of time ago and that's what the end output looks like so again we can go and generate more fashion if we wanted to and you can see we're generating generating a bunch of random and different types of images which kind of pass off as as fashion you could all even try to do this on colored images and see what those outputs look like let me do know if you innovate i'm always interested to see what you guys build thanks again for tuning in guys peace thanks so much for tuning in guys hopefully you enjoyed this video if you did be sure to give it a big thumbs up hit subscribe and tick that bell and all that good stuff hopefully you enjoyed this video where we sort of went back to the old tutorial style where we're coding up from scratch building up our models and then testing them out in real time thanks again for tuning in guys peace okay so neural network so the first thing that we need to do uh is

Channel: Nicholas Renotte

Views: 132,348

Keywords: deep learning, python, gan, generative adversarial neural network

Id: AALBGpLbj6Q

Length: 121min 23sec (7283 seconds)

Published: Thu Jun 09 2022