Coding Train Live: Autoencoders!

[Music] so [Music] do [Music] [Music] do [Music] do [Music] [Applause] [Music] do [Music] do [Music] check one two here is my microphone please [Music] i think i might be quiet you're gonna all tell me i'm quiet [Music] [Music] all right i have pumped up my volume a little bit so i have a feeling now you can hear me and i will be showing you my face in just a moment not exactly prepared for this but what are you gonna do here we are [Music] bye [Music] the train is pulling into the station it is me dan welcome coding train time i see my image over there i see chat messages scrolling by i see way better works for me uh from colby james says much better voice maybe still slightly low i don't know it is what it is there is a dial over here that i can turn to make myself louder but i'm a little bit hesitant to do it just at the moment uh welcome happy monday i don't have a watch on i keep looking at my watch to know what time it is monday 405 p.m eastern this is quite possibly the worst time energy-wise for me to stream um but it is the time that happens to work for me this week today although good news second stream coming this coming friday so this will likely be part one in part two and i just have to say so um long story but there is uh another dog who is staying with us who is not gloria who's behind a door over there and i hear whimpering crying her name is bailey i might have to let her out in a minute um but i'll see about that um maybe especially if i go over to the whiteboard so i've been working on my setup hopefully you can see the fruits of my labor some nice bright lights a really excellent keying of my green screen background i believe i can show you what it actually looks like without the key i had a little button for that but i don't see where it went anymore me green there we go so looking at this you would think like actually that's not so great it's a curtain i kind of kind of got to tighten it and some shadowing but good enough someday i'll have some machine learning model that will in real time just segment me out but until then uh green screen i also have been working on my whiteboard shot so let me attempt to switch over to there i think it looks much better i got a new tripod so the actual uh camera is a little bit higher um my my pants actually showing in the shot almost um and i've been working on focus focusing the lens so i definitely plan to draw some diagrams today let me know how this looks um could still work on the lighting it's probably very echoey over here because i have no sound blankets here and i still need to put up some sound foam on the ceiling but i'm there's a lot of work going on right now uh to get this studio going where i will be spending a lot of time uh starting this coming january um currently oh i didn't mean to show you that just yet but that's fine um i am hoping that by this spring the coding train will be entirely solar powered so wait and see we'll see if i can get there currently i am running off of the grid whatever power is coming to me from uh central hudson because i'm upstate in new york and hudson valley but i uh am working well i'm not doing this because i'm completely incapable i would love to do more but working on replacing the roof which is over my head in order to have something a little bit more a little more heft so that solar panels can then be installed and then i can power everything coding train wise this whole live stream uh and i suppose the internet connection that i'll be getting is sort of powered from some other location coming in through here but i'm getting way off too off topic i've got to get started on what i have for you today so um colby says i like this whiteboard setup personally works better than you might think says zero control and martin says i just decided to give up my tensorflow exam revision and watch this live stream terrible terrible decision no if you have real work to do with real uh studying of actual educational content i highly suggest you switch over to that i don't want anyone's test or grade or homework to be displaced by my shenanigans i'm hoping that whatever i do today will eventually turn into an edited tutorial video or a um something more than whatever it is i do today so let me get a couple of things i just got to jump right into the code stuff um unless um i don't i won't be doing community contributions or some of my other more wackier segments today francello asks is this a permanent studio well i have changed my studio so many times in the last bunch of years i'm trying to figure out there must be some like coding train anniversary coming up soon because i'm doing this for a while um but um this is what i hope will be a quite permanent studio and certainly uh you know i'm going to be my plan is to be incredibly active in working in here somewhat full-time-ish starting in january i have a little sabbatical from my job at nyu so get get used to it people you hope we're going to see more of me coming in january now um [Music] ah yes uh before i um get get going let me just thank today's sponsor of the coding train and are my friends over at brilliant do you like learning i like learning do you like watching me on youtube guess what if the answer is no if it's yes or no i have something for you uh brilliant um and the reason why is maybe the way you like to learn is through interactive lessons and interactivity and trying things out on your own so it's one thing to listen to me blah blah blah blah blah blah auto encoder neural network it's another thing to actually get in there and try it yourself i'm sure you know this from how it is to learn to code but brilliant is a website there is an app a wide range of courses all through the sciences uh computer science and physics and they have this brand new updated version of their logic course so i'm gonna come back in the middle of uh today's live stream and start the new logic course and see how it is you can sign up for free at brilliant.org codingtrain that lets them know you found it for me and they and that's a good thing all around win-win for everybody uh is free but you can also unlock to unlock all of the premium content all the courses if you use that link that's right above me right here you will uh the first 200 people to do so will get a 20 off of the annual subscription finally to monitor system is giving its best conceived video and chat file at the same time ah lee shala says what is sabbatical yes i have uh worked my hardest over many years teaching my classes seven uh years to be exact and as part of my teaching contract i get a semester uh without teaching any classes and to work on some research projects although i will say guess what my primary goal if i'm being perfectly frank is not the coding train now you might be like what you have a sabbatical and your goal is either first of all it's my own like mental health insanity it's probably my primary goal as it should be for you but um i have got to um get this nature of code book off by the way i'm just looking at my tripod and even though the camera is level the tripod is like so askew and then the top is like it's like is that not going to fall over like who puts this studio together this is insane okay i have to deal with that later that's really really looks like it's gonna fall over you can't see it everything seems fine to you i think it's gonna be fine nature of code book i have got to get that second edition of the nature code book out published new website new print edition new examples more machine learning i don't know maybe there'll be an auto encoder in there probably not not really part of that book but that's my number one goal is to i've said i'm going to do that the nature of code book actually came out i think 10 years 2012. so it'll be 10 years do you know when i started talking about doing a second edition of that book i don't even want to know 2015 at least so um so here we go all right so um i i don't i've lost my rhythm in terms of what it is i actually do on these live streams i'm trying to return to my roots which is essentially if i have something that i'm trying to learn or teach and i have not had a chance to explore that uh doing it while live streaming is the perfect thing for me to do so that's what i am going to do today and the topic that i am going to explore is an autoencoder now look at this building auto encoders in catas keras saturday may 14th 2016 what was i doing in may 2016 huh trying to think i don't remember let's see may 2016 a coding train can we find any videos that i was producing then may 30th object oriented fractal trees oh guess what i was probably doing then working on updating picture code booking p5.js oh that's so sad uh yeah look at this reaction diffusion ah those are the days when i used to do all these coding challenges it was another time back then uh anyway um i think i i don't know if i encount actually probably did not encounter this article around then i probably encountered it a couple years later a year later um this building auto encoders in keras is a guide that i'm going to use and you might today and you might be wondering like wait what what like what year is it on a meme insert here because if you have been following the landscape of deep learning algorithms to synthesize media generate images i mean you know that where we are today is something more like stylegand3 which is uh you know the latest and greatest uh generative adversarial network to um that can be trained on a data set of images and create new images in that uh style and uh we can all be sort of freaked out by the sort of hyper-realized realism of these faces that do not exist and um so yes that is true um i don't think but my knowledge i after i have this journey through learning machine learning my knowledge is kind of still like i'm still like just taking little steps forward from those old days where i was like let me try to at least just program in processing or in p5 and javascript and neural network and i think for me when i am working with one of these newer generative models let's say style i've never actually worked with style game 3 but let's say style gan 2 and i'm trying to understand wait what is this thing called the latent space why does that exist how does it work how is it that i pick these random numbers and pass them through here i get this image and then as i there's a fly by the way on my uh confidence monitor there go away fly as i change those numbers and then feed them in again how does all this stuff work so to me if you are interested in working with synthetic media working with gans to create images whether for art or science or whatever kind of creative expressive potential that those tools and models might afford you if you want to understand the sort of like core basic elements of how those things work just an auto encoder is an amazing place to start that's my point of view so this article i remember seeing it being like oh i think i understand what an autoencoder is kind of and look at this code i remember looking at it being like i don't know what um what any of this is keras what's keras uh so i don't know to what extent keras is still used as a term in uh the python implementation of tensorflow or the python bindings to tensorflow keras is a sort of high was or is a high level api for working with neural networks created by francois chole who is an amazing ai researcher and pioneer in a lot of the this stuff um it's made its way i work into tensorflow.js in tensorflow.js it's called the layers api i don't believe the keras word is there at all and someone again in the chat can tell me why it was called keras i think there's um i forgot why the name um and and martin is saying i follow that guide to create my first auto encoder as well the undershirt chroma key passed through is this undershirt being chroma keyed let me just see oh it certainly is that it's very distracting that would drive me totally crazy how about i do this whole stream okay hold on that'll help i was gonna like button the top button but let me just clip the lav mic to here i'm probably like way louder now too because the lav mic is much closer to my mouth okay i had to fix that jim that would have that totally would drive me crazy hopefully it might still be there a little bit um so um i'm just looking at this so i i lost my train of thought um but here we are auto encoders so a couple things one is i would like to before i begin attempt to record this session to disk i still am ironing this out but with i bought a new streaming computer got this big gpu in it huge hard drive lots of ram in theory i should both be able to be live streaming to you right now as well as recording i screen capture both cameras all as separate video files to disk so that if i wanted to this session could be chopped up and edited later i probably won't do that with this session but i at least want to see what the footage looks like after the fact so let me hit that button start recording um friendly viewers pedro asked is this going to be a python stream no so i'm going to do this um sorry i'm just looking at the uh member chat and discord i would like to do this i'm going to do this with tensorflow.js my goal is to eventually see if an autoencoder can be demonstrated with ml5 and the ml5 neural network which is a sort of high level layer on top of the layers api of tensorflow.js which all dates which is all you know keras is kind of like the big source of inspiration for um but we shall see all right so um ah so um uh christian vekman how did i do like i would say b-minus writes keras means horn in greek it is a reference to a literary image from ancient greek and latin literature first found the odyssey ah so i now i'm embarrassed more that i didn't know that because while i am not greek um there's a lot of greek in my family my wife is greek my children have greek names uh we like greek food haven't been to greece i think i was in greece in 2015. it was a good time that's what i'll do january see you all later i'm going to greece okay let me let's go over to the whiteboard and talk about um kind of conceptually what's going on here now one thing i should say is that and i'm just looking to make sure my audio is going um one thing i should say is that even though i've got this new setup which i hope the whiteboard looks a little bit better focus wise and lighting wise um still very shaky i don't know what i'm going to do about that buy a new whiteboard tighten this one up uh i don't as i'm standing here have any access to really being able to see the chat i can kind of see out of the corner of my eye the youtube chat going by so i suppose if something goes really wrong and now everyone's just gonna troll me if i say this but just flood the youtube chat with sirens emojis i might be able to see that um but at some point i am planning to set up a secondary monitor maybe over here or over here where while i'm at the whiteboard i can also see the chat and see what's going on but all right so let's start with just the sort of core let's start back to what is a neural network that's not a good place to start let's start back with what is a computer how about um i don't know where to start i want to just draw a kind of classic diagram of a neural network and let's think about a sort of classic image classification diagram so what if what i wanted to do was demonstrate that i'm saying classic way is it wait this is too classic for you uh there is the mnist data set right which is a data set of handwritten digits all 28 by 28 pixels meaning uh 784 pixels each so uh if you watched three blue one browns but what is a neural network or looked at this keras tutorial or anything else on the internet that's kind of like a basic like let's learn about machine learning and i i kind of have this like goal in my life which i am now completely failing at which is to never use amnesty and just try to always have other creative examples but uh such as life this is where i am right now if i wanted to build a neural network that could classify these images i would probably do something as follows i would have a set of inputs and i'll represent all of those as circles there would be 784 of them each one being fed essentially and a big fed is like a good way to put it by a given pixel okay so this pixel goes into the this input of the neural network here this pixel goes into it here etc etc etc so this is actually not a layer of the network these are just the inputs the first layer of the network might be called a hidden layer and of course there are i'm i'm kind of i'm talking about the most basic vanilla neural network uh right now so while of course if this is image data and i'm really doing image classification i might want to use convolutional layers which apply like filters and all sorts of other stuff and i have videos on that but right now i'm just sticking with the most basic idea of a neural network and in fact i'm going to make it so basic it's just going to have one hidden layer and this hidden layer would also have some number of neurons in it nodes units lots of different words are good for let's be really really uh unrealistic for a moment and just pretend it has just four now if what i was ultimately doing was the goal of this neural network was to take this input image feed it through the neural network and out the other side have a classification of the image is it a zero one two three four five six seven eight or nine right there's ten possibilities then the output layer would have 10 outputs i think that's 10. and as i like to do obsessively and any of my students who take my actual classes know if i draw if i draw one of these on the board i can't not fill it out by connecting everything this these are fully connected dense layers you hear that term and machine learning a lot what does it mean that they're dense or fully connected it means every unit of one layer connects to every unit of the next so this hidden layer connects to every single output and i uh you know this is just like you know okay is there like a fast forward jump through time thing i could do uh i just i mean it's like it's a meditation drawing all the connections the silly thing is i'm probably just in about like 30 seconds going to erase this so i'm not sure what the point of me going through this is but once i start i can't stop it's a problem at least i only picked four i should have tried maybe just picked three if i just picked three i would be done by now just wait till i have to do these okay i'm gonna stop maybe this is the first time in my life that i'm gonna manage to stop doing this i don't think so or the sirens going off in the chat okay i i i think i might have missed one okay don't you all feel better that i took the time to do that uh i'm gonna do just one input okay one like let's say this input is also going just also because i think it'll be easier to explain so now we have to imagine that every single one of these also feeds into each one of these hidden nodes so remember there are 784 of these there are 10 of these and so this is what i would say what is known as the architecture by the way i did not necessarily think that i was just going to start by explaining but i have to like warm up and get into this i don't have a watch on me i don't know what time it is i can't see the chat i'm gonna keep going this is what i would say is known as the architecture of the of the of the model of the neural network there are two layers a hidden layer and an output layer there are 784 inputs and then we have a process and the same process will be used in an auto encoder called supervised learning so the idea of supervised learning is let's say i have um oh i see sirens i think it's trolling or just for fun but i'm gonna come over here uh with hidden node three all right i'm just uh whoops uh this i um i think i think i'm okay i don't think there was a real siren i missed a connection says joseph i think that's what the sirens that's fair that's fair i miss the connection um all right so presumably there would be a a database of training images and each one of those images in the database of training images oh you know what an auto encoder you might consider like unsupervised learning in a way right interesting we'll come back to that because the labels don't matter with what i want to do so you might have like a bunch of images of digits right each with their label the supervised learning process involves feed one of these in what does the neural network guess like if you feed in the zero digit and it guesses it's a four nope that's not right go back and uh adjust the neural network according to the error and rinse and repeat so what do i mean by adjust according to the error i mean i'm hand waving a ton of stuff here i think i have probably a 10 part series that kind of goes through all of this much more methodically step by step and writing the code for it but the the thing that's important here is that each one of these connections is what's known as a weighted connection so there's a weight and i think an analogy that i've heard is that you can think of each connection like a dial on you and that weight can be tuned up or down you know what is the relative importance of this particular input um as related to some particular output and so as you feed this the training data through if i fed in this digit of a one and the neural network output said one there's no need to change any of the weights because it got it correct but then if i fed this next one in of a one and it gets to nine i've got to adjust the weights and the process of doing that with large data sets over long periods of time yields a sort of optimized set of weights that would in an ideal world will perform well with new data data that's not part of the training set there's so much more to say about this but i just wanted to sort of talk through the very basic uh um kind of process of what it means to train a neural network for image classification let me take a pause a short pause here and look over and see if there are any questions in the chat and i also want to um i also want to check what time it is because uh yeah um and um sorry okay i see that there's a discussion about shaders that was last stream i by the way i'm so obsessed with shaders and i'm i do have a fantasy of doing a whole like shaders tutorial kind of like learn shaders with me might be the title of it or something but um this has been this is part of the course that i'm teaching right now and i've been talking about auto encoders and generative models all semester and i really want to have a running example that i can kind of demonstrate so um all right supervised means we have correct output layers france francello yes supervised learning means that you have a training data set a training a labeled training data set so you are supervising the learning process with the input data and the correct answers can you complete your diagram second question good resources for fast iteration of neural networks and machine learning i don't know that actually sorry i'm reading the questions in real time i don't actually unders i don't know if i understand that question um but um hopefully the chat people can provide some good resources you know if you're interested you know not to plug things but i guess that's what i'm that's what i do that's what i mean you're doing ultimately it's all just a giant plug for me right no that's weird um join the coding train discord uh let's see if this will um that button didn't work but somebody will post it the codingtrain.com discord um you can it's a good place to have this kind of discussion uh lauren laurent strodio writes do you check all pixels from the initial pictures so um let me go back to the whiteboard here uh i believe the answer to that question is yes so i mean it depends right this is a totally made-up scenario now it's been you know some very smart thoughtful people designed this kind of scenario thought through it and i am here just sort of telling you how it works and maybe implementing it even with like a hot from a higher level with a library that has all the lower level details but there's no reason why we couldn't say like hey i got an idea let's see if it works with we just do every other pixel um but um so um so yes but um you know the max if we want the maximum amount of information from the input image we'd be looking at all the pixels there is something known as dropout which is a term in machine learning i believe that refers to the process during the training process of disconnecting certain connections right i talked about this being a fully connected dense layer we want everything to be connected but um uh you know you don't well there's a there's an issue that can happen which is known as overfitting so if the neural network is so good at producing the correct results for some training data set of images it might it might have learned it so well that it kind of can't perform when it's faced with new data and so allowing a little bit of noise a little bit of kind of like mistakes in there uh will uh could result in the neural network in the trained model having some more sort of flexibility a little bit of chaos if you will to be able to accommodate uh newer data and not be so rigid about its uh evaluations i don't know if that's a sort of helpful way to explain the concept of dropout or or overfitting but um you know i'm sure some people in the chat will write some comments with better explanations okay so i think i'm ready to move on to an auto encoder right now i'm just coming back uh to check the chat one more time um right uh okay wikman writes it's a random turning off of neurons during training thanks okay um all right so now the whole point of this is i'm not here to do this i have examples that do this uh and ml5 uh which is a javascript library that is a layer on top of tensorflow.js has ways to create image classifiers and other kinds of classifiers and regression models and all sorts of things it's ml5.neural network i'll refer you to insert here in post production all of the videos i've made about training your own neural network with ml5.js and if you want to just go and watch those now and leave the live stream be my guest um what i want to do is is is kind of talk about an idea what if okay what if instead of image classification let's take out this idea of image classification entirely um erasing sort of things have nothing to do with image classification but i'm erasing just to make this point and um and what i want to do is create i'm going to say a copying machine let's say that and by the way um i didn't watch it today but last week i watched the video from two minute papers which is called like what is an auto encoder so i would really recommend you just get turn off this live stream or come back to it later and watch this video or watch this video later if you're kind of invested in in my process here but um so apologies if i'm regurgitating any of the language or concepts from that video it just really stuck with me um but and i and i recall the video talking about a copying machine so what if my goal was to create a copying machine meaning these pixels these 784 pixels that are fed in to the inputs the outputs also have 784 units and i want to see with the neural network i want to see all the pixel values come in and the same exact pixel values come out you would be rightfully asking the question to yourself well that's ridiculous like why would i want to do that first of all i could write a for loop for i equals zero i is less than 784 and i could make an inputs array and an outputs array and just say outputs index i equals index input i right so the reason why i'm doing this is not to actually create a copying machine in the most exact and efficient way because with a low resolution image an algorithm to copy one image to another done finished two seconds turn the stream off go have dinner whatever go to bed i have so much stuff to do but um um so the reason but what it so so so why why why are we talking about this copying machine well with this process and i know i obsessively drew all of those but i need to erase them now for me to continue i'm looking for i don't have an eraser so i'm using this paper towel i'm so sad and um i'm gonna sort of mirror a diagram from i don't know if it's from that video or a different paper so what if this hidden layer right you think of this hidden layer you know i said it has four units to it what if there were some other hidden layers like here's another one and maybe this one has eight one two three four five six seven eight and there's another hidden layer right and maybe this one has eight right and these are fully connected layers oh it's not so bad it's only eight right i can do this if i start one i can't stop it's really weird like what is something is something strange things happen in my brain so you can get the idea that this is also these are also going and that these are coming in so the idea here like looking at the scenario is it possible to take this input image send the data through a neural network reducing the amount of information and then expanding the amount of information back out right this is a not only a copying machine but it is essentially something like a compression um engine or a compression right could we encode by the way and this is called this half is essentially called the encoder and this latter half is called the decoder could we somehow encode all of the pixels of an image into less data and then decode that smaller amount of data back into the full image now again uh the two minute papers video talks about this um this is not a uh particularly great great way to actually do image compression right if we want a smaller file size to uh for a lot of pixels there are lots of known ways to do image compression jpeg the algorithms behind jpeg compression being probably uh the most well known but the the point of doing this is not to have an efficient image compression algorithm rather if the neural network can learn how to encode essentially the features of an image encode the idea of an image it can then generate new images right what would happen the idea of an auto encoder is if we can create the encoder and the decoder in terms of like sort of this sort of creative arts world and lots of other things too but the idea is generative images what if i want to dream up new digits new cats new whatever i think i'm going to create a data set in processing and just like draw a bunch of shapes so if i want the neural network to be able to dream up new shapes could i train an auto encoder with a data set of images then essentially slice off the encoder i have this trained model and start just from here feeding in noise random numbers vectors latent vectors so i want to get into this idea of the latent space latent vectors um but uh i think i'm kind of like i'm trying to explain this i've sort of reached the point where i think i need to start writing some of the code we could double back to this diagram and kind of get into the reality of how it works but this is the idea and i feel very disconnected from you because i'm not looking at the chat and all of that but this is really exciting this is the foundation i mean again works in a very different way there are two neural networks but ultimately the idea of a latent space generating images and the ability to navigate and experiment with that latent space to generate animations uh um um of those generated images there's so much there to explore and auto encoder understanding it building your own auto encoder i think is a great place to start to see how all of the pieces work um as you then as you then later find yourself just running collab notebooks big models and different things um learning the getting used to this process understanding the vocabulary of it will give you a leg up in kind of working with more cutting edge research into generative models let me see what questions they have or if there are other things i mean the other thing i was going to say i think this also has some pretty interesting applications because if um so denoising i think it's mentioned on the the keras page that i want to look at so imagine if we train this auto encoder to copy images essentially so it knows how to look at an image of a one and kind of recreate that same image as best as it can if if to learn to do that very well and i pass in a very noisy image it's like fuzzy it's got some method like just grainy extra random noise in it the autoencoder is trained to generate non-noisy digits so it it can actually work as a denoiser because it will take that input information and try to create its best approximation of it but it's learned how to do best approximations of non-noisy images so its best approximation of a noisy image is hopefully going to be that that the content of that image without the noise so that's one application of this as well that you can think about let me go over to the chat see what's going on and see if we can get started writing some code i also need to take a break and look at the brilliant logic course um shortly um so hold on i'm seeing some interesting uh oops and this computer went to sleep i'm okay i'm seeing some interesting uh comments in the chat here uh mark booth says really want to understand but it gets so abstract so fast sorry uh uh lee writes got an idea put the board behind the camera you're normally at and then you can look to the right instead of the left put the board behind the camera but i don't oh i go over there i don't know i have to think about that okay so um hopefully this has been helpful to folks i'm looking at the time 4 45 all right i'm gonna start let's so this is my this is my plan i am going to generate a data set i'm going to get my node.js tensorflow.js environment setup take a short break and then start come back and start writing the code for the autoencoder itself so let's open up processing um i you know i what i would like to do is create a data set of images one that is pretty sort of simple to work with okay so let's call this i'm going to save this onto the desktop um what size is my font that looks good let's save this as a auto encoder training data all right so i'm going to write a processing sketch by the way look at the beautiful processing four gotta love that uh and let's do the following so i don't know what would be so i'm gonna work with 28 by 28 images like really low resolution images you know soon we can get everything working we can try to see how much far we can push it with more training data higher resolution um so let's just actually i'm going to though what i think what i'm going to do is i'm going to make my processing window 280 by 280 then when i save the images um i'll i'll sample them down just so we can see so let's make a black background i'm just going to do rec let's just do squares so let's pick a random value between uh so um i'm going to say uh wrecked mode center and uh stroke weight 4 and then if i said square width divided by 2 height divided by 2 100 let's just try that let's get rid of this right now so there we go there is a square so what i um i think i'm going to make it a little bit thicker uh let's make the stroke weight eight because again i'm going to have to like sample it down i mean why not make it 16. so i think what would be interesting is to ultimately do a data set that's many different kinds of shapes but let's just do squares that's 100 pixels so let's have a value between 100 and 200 and use that as the width so this is sorry for the flickering but now i want to save all these images so but i want to say and i'm just going to say no loop just to do one right now so let me and i'm sorry that i'm uh coding standing in front of the code so we move this over here so let's do i think i can use get p image image equals can i just say get save square dot uh should i use a png pro or jpeg i don't know is this going to work let's just see what happens so i run this let me go to the folder with this processing sketch yeah so that worked but the reason why i wanted to get it make a copy of it is then i should be able to say image.size resize i think this is right so basically i want to draw something in processing um then i want to just create a copy of that image resize it down and save it to the uh so let's look at this yeah you can see it's fuzzy looking because it's now a very low resolution 28 by 28 image fantastic um so i want to do this multiple times and i gotta processing this is really nice feature if you use save frame it will like number all the images as you're saving them but because i'm copying it using save i can just use frame count though so i think i could say plus number format frame count four i think number format will always give it four digits you know i don't think i'm gonna make more than a thousand so let's just do three plus dot png and let's put that let's make a folder called data so this should be saving every image in a data folder numbered according to the frame count always using formatting the number always to be zero zero one zero zero two and let's see uh let's see what happens i mean it's going to bother me that the first image isn't zero let's run this apologies for the flickering oh no i have no loop so let's see yep square oh that really drives me crazy that it's not square zero zero zero i'm sorry i have to use my own counter i mean uh let's do a counter equals zero let's change this to counter let's say counter plus plus and then let's just do 100 images if counter equals 100 exit so now i'm going to run this sketch this is a really simple processing sketch just to generate 100 images of squares to train our auto encoder and obviously i'd like to do something better now so i'm like taking a look at the chat okay so let's see what happens here right we should get 100 images i exited let's look at the data folder and we can kind of scroll through this and we can see here are my 100 images all right so that step one create a training data set of images just squares i really want to also make them circles no so one of the things that i have learned is that as tempting as it may be right now for me to uh build a more sophisticated scenario um example i just want to know that my code works even if it produces a sort of obvious and trivial result so if i can get whatever i build to produce a very obvious trivial result then i can then i know that things are in the right place and working and i can try to start exploring with more complex data and more complex scenarios right so next step so step number one is create my training data image set and i'm looking at the member chat um yes yes yes yes yes um simon is giving me some good advice in there i'm sorry i don't have my the discord is like really really small so i don't see it too well today apologies for that uh okay now what is next what is next you know how people used to make all of these like weird musical remixes of my videos that doesn't happen anymore that's the internet did tick-tock happen and nobody wants to make weird musical remixes of my videos anymore come on people it brought me so much joy it brought the world so much joy [Music] okay um what was i doing i believe ah okay let us go now to i terminal window okay so i'm going to go to the desktop i'm going to make directory auto encoder tf demo all right so now what i want to do is write the code for the auto encoder now ultimately what i would love is to someday open up the p5 web editor and like demonstrate the auto encoder running visualize the training process maybe uh then generate new images have like sliders to play around with all the latent vectors but i'm at the beginning of my journey understanding this and it's it's sort of i don't know whether it's fortunate or unfortunate for you but what i often try to do with my videos is come from a sort of high level place and use a lot of other tools to demonstrate a concept and offer the audience a chance to explore that concept with their own creative in their own creative direction so i think eventually it might be nice if i kind of have a lot of pieces ready to go in a video that can be right at this part where people can kind of make their own creative version but i just i want to understand how all this stuff works i think for me being able to just work with node and tensorflow.js directly without any interface will allow me to sort of figure out the um um allow me to figure out how how the autoencoder stuff works and then i can come back to later to see if i want to do it in the browser with more sort of visual flair so i'm going to create a node project by saying npm init then i'm going to like just like hit enter enough times that the fact that i just hit enter and didn't think at all about what i'm doing good um let's also hit git init um to oh can i change look at this git config dash dash global init default branch this is new main i want my default branch to be called main to match what github does main trunk development and then i could say git branch dash m main right there we go okay great so now i'm in so i also want to just keep this project as a git repository because that way i can like track the history of what i'm doing and then you know at the end of today certainly upload what i have to github and then come back on friday to complete this project um so what do i need i need tensorflow.js uh all right so import blah blah import blah blah blah import blah blah tensorflow.js for node okay but i guess i do tf.js-node uh okay so let's see so i want to install the tfgs node package error no not what it's called you can see by the way that i in case you were thinking like did i do all this earlier and now i'm just rehashing it clearly i did not um oh oh i think i call it i have to do this there we go so i think i am now installing tensorflow.js node is it time for me to switch over and use um es6 imports um i think it might be so i'm going to try doing that i'm not going to do the gpu i'm not going to do this with any gpu stuff oh this might take a little while because i don't i didn't okay so also um let me open this up in visual studio code uh yes so here i am and all i have so far is the package.json so you can see this is the name of my uh project the version might call this version 0.01 and maybe i'll give it a description coding train attempts auto encoders so i recognize that some of you watching might not be familiar with node.js or node programming this is not as sort of friendly as me just opening up a processing sketch or a p5.js web editor but ultimately i'm creating a project a software project built in javascript this package.json file is the sort of uh configuration of the project um it's telling me i don't know why it wants to do this that the it's telling me that the file to run the code for this project is called index.js and one of the other javascript libraries it's using is tensorflow so let's create an index.js file let's just say console.log hello auto encoder and then let's go back to my terminal and i'm going to say run it by saying node index.js and i'm seeing that console log so i am executing javascript code from the console via node.js and this is going to be this is because i'm importing the tensorflow library i'm going to be able to write in javascript hooks into the tensorflow machine learning library in here and program my auto encoder and in theory like i should be able to uh to some some sense just kind of really follow this and like port this code um which is sort of python keras library into javascript with tensorflow.js okay close this um but where am i um so i do i think i want to uh just as an experiment so typically what i will do is next say something like equals require and i don't even know if this would work would this work like this is what i would typically do next which is require the package and take basically everything from that library and put it in my own variable i'm just curious so that seemed to have worked i'm curious what this uh weird messages is giving me tensorflow binary is optimized with one api deep neural network library to use the following cpu instructions to enable them in other operations i don't i don't know so this is telling me about how i built tensorflow and maybe i'm using cpu and maybe i could do it a different way to make it more efficient but i don't actually care about any of that because i'm doing some a very much of a toy example with just a few images at very low resolution but i am kind of curious i you know this is not germane to the tutorial but i'm just curious what happens if i change this to an import and then i think if i do this it's going to say cannot use an import statement and so to use import statements don't i do something like something module uh package.json es6 module import what do i put in it uh what do i put what do i put input type type type module there we go so i can add type module and this is so not important to what i'm doing right here but great so if i add type module to package.json that tells the [Music] node engine that i want to use import statements instead of require which is a newer fancier better way of doing it i think you know this is coming from somebody who refuses to type yarn no npm npm that's what i've been doing for 100 years and i will still do it tomorrow npm and then like i like what i like um but i do sort of the one thing i think that's important here is i'm on node version 16.11.1 and um [Music] i think the import syntax is only supported in node version blank and above um type module okay so again not super important and simon says yarn isn't as good anymore anyway see if you wait long enough and never bother to use the new good thing it eventually it just goes away gets replaced by something else lessons deep thoughts or lessons about programming with your host dan schiffman don't bother learning anything new because if you eventually just wait there'll be something else new that you need to learn if you don't bother learning that that'll go away too stick with java never going away people all right all right i'm looking i'm looking looking i'm looking v13 is the minimum version for es6 imports okay it took you three years for var to let yeah um all right so now i believe i am set up and ready to go um oh there's a lot to be done here all right i'm trying to think this is probably a good pause time oh the couple things one let me take my images this data let me go to um this folder reveal can i do like reveal come on reveal in finder let me put this data folder here okay so um i'm actually gonna i'm gonna throw this project up on github real quick so people can take a peek at it but one thing that i want to do before i do that is um i don't want in my get history this images to be there so i'm i'm going to rule if you are someone grabbing this code and wanting to run your own version of it um you'll need to generate your own images or download sample images from somewhere else i don't want that polluting my git repository and git history and i also don't want the node modules so i am going to create a dot get ignore file and i'm going to put the data folder in there or maybe i could also just do star.png there's different ways i could do it but um i don't know which matters i'll just do them both i don't want the node modules folder and um i don't have any environment variables yet or anything like that i mean i could add that in just in case these are some sort of standard things i also like nothing bothers me more than the mac os ds store files so this is some pretty good git ignore stuff i'm going to say git status just to see where i am [Music] let us add all of this stuff and i'm going to say initial project setup um let's go to uh github.com oh no i'm not signed in if i'm not signed in this means you're gonna have to wait about 45 minutes for me to do like all of my various randomized passwords lookups and two-factor uh but oh no look it's saved all right it's not so bad so i just have to do my two-factor i forgot that i did that in my for my streaming account so let me just get my um authentication code i'm gonna just move over to here no reason for you to see me type it in um one i don't care you know the first digit is one uh okay i'm in i'm in uh i am going to make a new repository um let's put it in the coding train org auto encoder demo public uh create the um code from live stream uh started what's today 11 15 21. let's create this repository and i can uh just add you know i don't need to do this is showing me instructions about how to link my local repo on my computer but i've kind of done uh uh done oh yeah i could just go right here so i can just add this repo as a remote and then i can send it along i don't need to worry about the branch i've already made my branch main and so now uh if you are following along i need to add read me and do more um the code so far the project configuration is here on github put the whiteboard facing the green screen oh but it would be in front of the camera oh behind the camera so i'd walk over there all right the camera's actually quite far from me like if you don't realize i have to walk all the way around walk over some cables and stuff and then here i am at the camera because i like a wide sort of view but anyway i'm taking your suggestions putting them in my mind uh so it is 507 i am going to be streaming until six o'clock so i think we have some time to get started on the actual tensorflow code and i will be returning this friday to completely this is by the way this is the new dan i'm just gonna see how i'm gonna give myself a lot of time remember like i'm gonna do shaders and auto encoders and look at community contributions and then i'm going to also uh demonstrate this whole other fractal pattern that i just learned about yesterday from watching a three blue one brown video all in 45 minutes no i'm starting this auto encoder project getting as far as i get today i'm going to turn this off and come back and i could continue where i last left off that's the wheat this week is the auto encoder week but before um i get any further i would like to take a moment to i'm just opening this up here thank i have some notes boy my setup is really not the best i would like to thank it's coming i'm going to press the button it's all going to work today's sponsor brilliant whether you're looking to learn something completely new brush up on a few topics uh you're interested in stem there's no better place than brilliant it is an interactive stem learning platform that helps you learn concepts by working through them in visual hands-off ways hands-on look at me reading a script i'm so bad at it fired always fired let me tell you i know that i'm reading for the script here but i actually really love brilliant and one of the things that i love about it for me right now is that i do it with my kids because my daughter is learning how to like do factors with numbers i was like oh look there's a course on factorizations little interactive lessons and we can do them together there's so much awesome stuff extensive course catalog constantly expanding um and you know some of my favorite courses i'll bring them up here one is the oh right there's a whole course on neural networks so do you remember this ridiculous uh drawing that i was trying to do to explain neural networks look at that look at that look at this on uh in the brilliant neural network course you can actually read about understand interact with neural networks as they recognize digits right there in real time so much to explore with that the the logic course is one that they have just completely is all brand new so it's a revamped edition has new lessons challenges and a much higher level of interactivity so um what i would like to do what i always enjoy to do with my sponsor segments rather than kind of play these pre-rendered clips is head over to brilliant itself and let's take a look um at the courses i'm going to um yeah what is a factorization is in there pre-algebra there's also a great what is a derivative course if you're interested like if you want to like learn some basic math stuff there's really really great okay so let's go over here sorry i just wanted to be able to see the chat again i lost the chat um so let's find um the logic course so i'm going to click here on the logic course it says continue because uh i clicked on it earlier today and then i was like whoa this is way too much fun not gonna look at it gotta save it for the stream so let's go to continue and see how it works okay so let's try this one first rey nook and lex are robots from and let's see if i can make this a little bigger for you and that's hopefully going to help a little bit are robots from three different models rey is not the newest knuck is the oldest lex is not the oldest so what's the correct order from newest oldest i think i'm going to be able to do this one um by the way i love these little robots just like as an exercise like could you make your own little robot characters in p5.js so much inspiration here okay so knuck is the oldest so nuck has to go here uh rey is not the newest meaning it's only other option is here so lex is the newest and i'm gonna check yes did you hear that little ding by the way it went in my ear i don't know if the sound is going out that was exciting okay let's hit okay go to the next one arrange the robots to make the following true loy is in one of the two middle spaces mig is to left of lex and right of rey rey is directly next to mig so let's just try putting loy here and then is there anything okay so mig is to the left of lex and rey is is directly next to mig okay hold on so what happens if i were to do this if meg is the left of lex and rey is here ah that can't work because lloyd's got to be in a middle space oh is left of mex and right of ray oh but lex could be here and loy could be here so are there are there more possibilities so this this satisfies right loy is in one of the two middle spaces meg is to the left of lex and right of re and re is directly next to mig i have a feeling if i did this yeah it wouldn't work right because then we couldn't have rey directly next to mig so let's check okay i got it right i don't know how many of these i'm going to do but this is like i mean are you not having fun i'm having fun one of these i'll get wrong eventually three robots compete in a race marv and lex do not place next to each other so the thing is first of all i should know about this because uh many many many years ago i was a i majored in math and philosophy as an undergraduate wasn't a double major it's like a combined major and what is what is math and philosophy together combined is one thing it's the study of logic so what i love about this is i'm kind of just enjoying this right now as a puzzle as a game i mean this is kind of like the stuff i love to you know you know i don't want to name other kinds of puzzle activities i like to do on my phone but like uh brilliant being one of them i like to just solve puzzles and play games um but what you i don't really what you don't realize maybe is that um i'm learning all of these really important foundational concepts in logic and i have a feeling that as we go through this course that will be revealed three robots compete in a race marv and lex do not place next to each other rey beat marv so if rey beat marv i think the only way for this to be true is this configuration because rey beat marv and marv and lex are not next to each other so the only way for marv and lex not to be next to each other would be either ones in third and one's in first but the other way around but for lex but for ray to beat marv it's got to be um marv's got to be in last still correct eventually i'm going to get this wrong okay in the next race give myself a few more minutes oh i think i can maybe get to the end of this let's see if we can get to the end of this um because let's see in the next race one robot finishes between rey and lex two robots finish between lex and ty so if two robots finish between lex and ty they one has to be first one has to be last and one robot finishes between ray and lex i think i might have just gotten it because i mean i got lucky but let's try doing it the other way right if ty was first and lex was last rey didn't finish second but two robots wait oh no wait hold on one robot finishes between ray oh yeah yeah but so tied in so then there's no robot between these two so tie has to be last yeah and marv has to be the other thing about this is honestly like imagine programming your own interactives like this so while this is a little meta for you the coding trained audience i'm emphasizing again like so many of my coding challenge ideas have come from working through these brilliant lessons uh got it right okay we're almost there um arrange the racers in the order they finish okay rey isn't first and tai isn't last okay lex doesn't place next to ty or rey mar finishes the head of ray so my strategy for doing this is trial and error like let me put one in that because it's interactive i wonder i'd be curious here from you what are some more methodical ways of doing this but rey isn't first so re could be last tai isn't last so tai could be first probably not right lex doesn't place next to tai or re nope so so lex would have to be like ty and rey have got to be in like the first two or the last two in order for lex to be not next to either of them and then marv finishes ahead of rey so i think this satisfies everything um so rey isn't first rey is last lex doesn't place next to ty array correct marfish is ahead of rey so i think i've got it let's check by the way i like seeing their explanations um and seeing the explanations we can see from the second clue we can conclude that only marv finishes next to lex so either lex finishes first with marv second or last with marv sir so the explanation is following something similar to me there's only two possibilities let's try one the third clue tells us marvin is ahead of rey so that means lex must finish first with marv second okay right then you've eliminated with another clue okay back to problem let's go to the next one okay three more the final race is between five robots oh boy this is getting harder now um let's see rey finishes faster than mars the smallest difference in finishing times was between rey and lex the largest difference in finishing times was between lex and tai well that means doesn't that mean either lex has to be first and tie last or vice versa right smallest finishing time just means that rey has to be next to lex we don't know the difference but obviously that if it weren't next to lex something else so this is this is these two give us this possibility or i could put lex here and re here mig finishes in first or third so mig could be third marv ray finishes faster than i think i got it luckily so i think i had a 50 50 chance and i got it luckily i think let's see got it correct i'm really rocking and rolling here review and reflect the key see this is what i mean i was just having fun playing with a logic puzzle working my brain um i wish my kids had been here to do this with me because they would have really enjoyed this or they might have actually been like oh my god dad again with the brilliant puzzles but they like it i know they do the key concept from this lesson is start where you have the most information given multiple pieces of information start with the one that lets you deduce the most the next challenges are a bit harder recognizing useful information is the skill you will hone throughout this course well i want to hone that skill for sure the next challenges are a bit harder they'll require you to process more information and recognize what's most useful all right i have completed five percent of the logic course solve problems by visualizing relationships did i like this lesson oh yes i did continue abstract order logic strategic order logic so i um next time that i am doing a live stream sponsored by brilliant uh i might be much further ahead in this if i've been going on on my own otherwise i will return to do this um if you like what you see here um and want to try brilliant out yourself um please uh take a minute i'm gonna take uh just a short break just for a couple minutes i'm looking for oh it's already there you can sign up for free at brilliant.org codingtrain that lets them know that you found uh brilliant from me the coding train yay that's a good thing um and uh there's a lot that you could do on brilliant for free there's uh daily puzzles and different things if you want to unlock all of the premium content all of the courses um then uh through that link you will get a uh the first 200 people to sign up to that link will get a 20 off uh description it's not it's not really the holiday season just yet but i might also just make a note that buying somebody a brilliant subscription as a gift is a wonderful thing i mean a it's not just more plastic nonsense it's educational and learning and fun i mean that's like that if you want to i already have it but if i wanted a gift that's what i would want can you get me like the uber super awesome extra uh premium brilliant then that you get me as a gift okay um thank you brilliant for sponsoring the coding train for keeping this engine going for allowing me to uh set up this new studio for helping really make all of the all my dreams come true you the audience though i love you all i'm going to take a short break i'm just putting up this intermission card here i'm gonna try to just limit my break to about uh a minute or two um put on some music here um just um so i can check and then i will be back uh in just a minute or two so i can keep working on the um auto encoder project see you in just a minute or two [Music] [Music] oh [Music] so [Music] so [Music] [Music] do [Music] [Applause] [Music] do [Music] do [Music] okay pretty short break i'm just getting myself back to being set up [Music] and let's get my code here [Music] all right all right i suspect that um nobody just happened to join at this very moment but if you did i am working on making a tensorflow.js auto encoder example i spent the first half of this live stream and by the way my the boiler in the garage is running so apologies for that extra noise but i spent the first half of this live stream kind of talking through this diagram oh boiler went off um in terms of what a neural network is and what an autoencoder is and why we might want to try this and where how this is going to lead to understanding latent vectors and all of that kind of stuff so i am now right here wanting to work with tensorflow.js in um node itself um so i need to start building the autoencoder using tensorflow.js code so um let me go to tensorflow i'm going to look at the tensorflow.js api docs so i haven't done this in a while and i want to use the layers api so the layers api is essentially and sorry this is so small um if i understand it correctly is essentially the same as i can't get to the right place okay um the layers api is the same as the original keras just without the name keras so the first thing that i need to do is create a sequential model so i'm going to do that right here so i have imported the tensorflow library i'm using the node implementation of tensorflow.js i am now creating a model and i is do i dare i rename this to be called autoencoder is that bad for some reason to me that feels good because what i'm doing is creating an autoencoder it is a sequential model in the sense that uh it is a feedforward neural network the layers are um architected in a linear sequence sequential that's why the word sequential comes there the next thing that i need to do is start adding layers so the way i like to do this um is i think i'm going okay so let's first look at this um this this is gonna i would love to sort of like read all of this out loud for all the explanation hopefully i gave something of a helpful explanation but let's look at this so the first thing that we need to do is create the input shape which has 784 pixels a lot of people are asking about shaders in the chat is it just because i did shaders the last couple times or is there something straight like still in the thumbnail or the description that's mentioning shaders that i forgot to take out so not today for shaders unfortunately so um interesting so i want to create the encoder layer and the encoder layer had in this example has 32 units so how do i let's see basically i want to write code that's like this i want to create the encoder which is a tf layer that has like 32 units and the input i'm making this up i don't think this is right is like 784 pixels or something like that so i don't recall the syntax and then i want to say auto encoder add layer encoder like this is basically the idea um clearly i've gotten something a tf layer like is that a function i don't know so i'm now just like making up code like i want to create this layer right i'm creating this layer this decoder layer that is going to receive 784 pixels and have 32 units so let's see if i can actually find the right syntax for this in the tensorflow documentation so going back to that which is here tf layer is that like is tflayers.den oh look at this all right so i was pretty close like if i just put this in here so i want to create tf layers.dense so this is a function that creates a dense layer with 32 units and the input shape is 784 so input shape okay great so that's the encoder now in the original keras example i just make oh just one ah so so this is so we're gonna i'm gonna definitely want to expand this but this very very basic example is just two layers the a hidden layer and output layer essentially the hidden layer is the encoder the output layer is the decoder the inputs remember are not a layer that's the data being fed into the first layer so on a very very basic level here i'm then going to create the decoder the decoder which has 784 units now interestingly enough ask yourself this question what is the input shape to the decoder i should like put a poll up and have people vote on it like is it 784 is it 32 is it something else so remember the encoder has 32 units we're putting 784 pixels into it the decoder has 784 pixels and we're decoding what those 32 units so the input shape is 32. however i believe that this does not need to be stated because tensorflow.js like i must specify the input shape for the first layer because how is it going to infer that i mean maybe it could infer it from the data somehow but that's ml5 kind of does that but that's not going to happen here but the decoder can infer the input shape if i add it directly after the encoder so i'm creating my auto encoder model it has two layers an encoder which receives 784 pixels and a decoder which pulls from the encoder to decode it to 784 values um okay yeah and uh um eat in practice and i'm repeating uh okay weekmans from the chat like in practice you would have multiple encoder layers and multiple decoder layers i'll add that in if we can get this to work okay input image okay i'm not going to worry about that so this is right so compile so i forgot how to use tensorflow.js so i have video tutorials on how to do this but i'm going to just right here so what is the next thing i would typically need to do is once i create the model is just looking isn't there like a i should look at my old examples right i have examples where i do this kind of thing like tensorflow js coding train github and these are probably so way out of date but like i need a bunch of uh examples like if i go look at this doodle classifier for example we can see i'm adding a convolutional layer with all sorts of stuff and then oh right then i have to do this i have to compile it so let's let's grab this code here and see if this is kind of the right idea so this is just setting up the layers compiling the model what is that doing well it's connecting all the layers and setting all the functions that are the math for how the elements are processed um but if i if i if i look at this right this basic example this is such an old example right the activation function is just sigmoid so oh and this one is relu which is not at all french but i can't help but say it's a french word so if i wanted to follow this also i missed that i could say uh is it um and apologies for not taking the time to explain activation functions although now that i said that i think i'm going to have to just briefly touch on it but i think i can put the activation functions in like this right just looking at my old examples yeah activation right um okay so what is an activation function so if i go back to my diagram and there's a you know ton more to say about this obviously and some of my other video tutorials will go through this in more detail but um these dense layer connections right there's some number is being outputted from here and outputted from here and remember there's all these weights so the all of the values all of the stuff that's coming into this particular neuron is the weighted sum of all of the the the units that are connected to it in the previous layer so all of the output values multiplied by their weights all added together get get processed inside of this particular neuron and how do you process those things with an activation function basically the um the way to think about this is if you know if you think very loosely about you know a you know i should say like a fictional version of how our brain works because i'm obviously not a neuroscientist and i'm not going to speak with any rigor here but you know oh my brain all the neurons are connected and they're lighting up or turning off and i see this delicious blueberry and i smell the blueberry and those are the inputs and all my neurons start lighting up right so does the neuron light up or not does it activate so we want to process all of those numbers and process them through a function which will essentially like squash that weighted sum weighted sum could be some big number small number into some range like between zero and one or is it just a positive or negative number is it on or off so that activation function sigmoid you know it's way it's this now is not the time for me to start going through the math of different activation functions and why this one is used and not that one is used i think um this is reading that i would encourage you to do maybe some people in the chat could offer some resources i certainly touch on it in other videos of mine but um sigmoid is a sort of like classic if this is all about classic machine learning sigmoid is sort of like the classic activation function not used very much these days but let's use it just to match this particular example then i was remembering i need to create an optimizer with a learning rate okay so i need to compile like this is just the sort of architecture and i need to tie everything together what's a good way of explaining what compile is doing i mean understand what it means to compile the code but i don't know that it's i don't i'm kind of struggling to find the words to describe what it means to compile this model after i've set up all the layers but a learning rate is there a learning rate specified in this particular tutorial learning rate space rate no so probably it's using a default one let's just keep that an optimizer i thought i saw in the code here the optimizer is atom okay so let's match that i think i can just do this atom i think actually i probably could just do this but um i'll do it this way and then the loss function is binary cross entropy so let's and metrics i don't know that i let's just comment this out for right now so these are you know i'm i'm kind of at the point of the day it's like 5 40 p.m eastern my brain is not uh functioning very well i need to eat dinner i'm tired i don't know what to do i've gotta go teach my class in person tomorrow um i didn't finish this but um i'm gonna i'm gonna do my best here so um you know when i come back on friday maybe i can come back and sort of talk through some of these elements a bit fresh but essentially the loss function is the mathematical function that's going to calculate how well the network is doing so in this case if i'm trying to create a copying machine how closely do the output images match the input images a loss of zero would mean they're exact reproductions and binary cross entropy is a particular kind of loss function like what would be something what did i have before there would be like a categorical cross entropy i'm trying i'm trying to just think of what um it's escaping me right now but um uh root mean squared error or something it's what i'm i think is what i'm looking for okay sorry uh let me comment this out okay so let's just see if this even runs like this is going to allow me to build this model compile it you know i haven't actually done anything yet but let's let's just try running this node code see if i get any errors okay i got an error auto add layer is not a function i just made up a function called add layer let's go back to the api add it's just add okay great simpler so it's not add layer it's just add okay uh unknown loss binary cross entropy so that is not the name of a loss that is in tensorflow.js interesting so let's look for tf loss loss hold on what was the one that i had in my other example because i can search for it categorical cross-entropy so let's look for it no whoops sorry on this page ah so much going on aha here are the here are the loss functions ah so binary cross entropy is the one i'm looking for um so i can just uh i assume i just wrote it incorrectly because i'm kind of that's like the python way of doing things with an underscore and the javascript way of doing everything is with camelcase so let's see if this runs with no errors now okay hello autoencoder so i realize that those of you watching this if you've been really following this whole thing this is very much going to be a 2b continued operation i mean i do have 20 minutes and in previous live streams you'll see me scramble be like i got 20 minutes i'm going to like try to just jam it all in there and get it to work not doing that today um so um but i'm gonna let's see i do have a little bit of time before i wrap up let's see if i can get a little bit further here so let's look at the keras code so i need to somehow figure out now ah so i could just create like noisy data what if i just made noise like random data and see if the auto encoder could produce random data would that be interesting or should i try to first import my uh images hmm well the next thing is yeah so this is what i need to do next um oh interesting i'm like what is going on here this is so crazy i'm like where's the y's where's the y's where are the y's i just realized you know what i think i can do this with fake data uh in the next and then the next stream will be actually the next stream will be actually using the real data so i'm reading some interesting comments without activation functions you will get pca components principal component analysis add gaussian noise then denoise interesting input noise yeah so let's just for a second let's put the let's try to create some training data so this would be known as x train so just for a second here just for um let me um let's create um a let's write a function random um i'm trying to think of what to do here let's consider an image to be an array of 784 values each value being a random number so this would be generate image it's not really an image wait why is my auto format not working okay so if i were to say uh x train oh x input is an array x inputs index i equals generate image so this is me creating like 10 arrays of 784 numbers and kind of unnecessary to put that in a second separate function and i'm not really being thoughtful about how i'm organizing my code so this is the training data but that's just it as a raw like if i just do console log console.log x inputs and run this code huh so i have a mistake oh return image and let's just make these with seven values just for simplicity's sake for a second right these are now my this is my training data and i want these images to be the image eventually these values to be mapped to the images from processing but i'm not there yet so i just have array array array 10 of them um so but they need to be 784 and the thing is they cannot be raw javascript arrays tensorflow works with tensors a tensor is just a bunch of numbers with a shape right is it an array of arrays there's an array of an array an array of arrays how many elements are in each array of array right um you know a an array is a list of numbers a matrix is like a two-dimensional array a tensor is a much more sort of generic view of of this um and can you input your logo that would be interesting to hear um to try so i need to convert these plain vanilla javascript arrays into tensors and i believe the way it's like re it's doing this like reshape i don't remember how to do this so i can look at the um ml5 does oh when i've been working with this all this for you i bet you i did it here in this example um classifier batch size data reshape yeah batch data oh my goodness reshape fit oh where am i making the tensors in this example oh yeah there we go tf tensor2d yeah so that's this is much simpler than i so the x train could just be tf tensor let me just paste this in here yeah tensor2dx inputs now i think like i could give it batch sizes and all sorts of things but i think it's going to infer things correctly and if i just say x train dot print let's see what happens so this is basically i generated these 10 arrays of 784 numbers and then i'm looking at my watch that does not exist it's four hairs till i have to go and then i'm creating a 2d tensor it's 2d because it is an array of arrays yeah this looks good look at this my memory of how to work with tensorflow is really um is really really coming along here i'm i'm enjoying seeing some of the nice feedback in the chat um from ali thank you so much from anurag i don't know if you're referring to this class or something else but thank you um and i know that i'm kind of standing in front of i need a little like to move this over here okay so now i should be able to go back to essentially doing the equivalent of the fit function so for example this is python but i got to do this in tensorflow.js i'm going to fit x train and x train and then my assumption is like these so this is really really weird x-train x-train i'm gonna come back to that this is why it's like is this supervised learning or unsupervised learning it's kind of interesting to think about um but then i assume i would do have an object that would have like epochs 50 batch size uh two like 256 shuffle uh true i'm not going to worry about the validation data right now that's probably really important but so i feel like the equivalent must be this now this is um and this must be as well is this async an asynchronous function tf tensor fit is so let me look at model.fit the fit function x y args fit compile oh right so look this uh a weight model dot fit batch size epochs model dot fit the x's and the y's that's a weight but to create a tensor the tensor is synchronous so this is asynchronous so i could use the then but i'm going to do this i'm going to write an async function called train model and i got it the format on save is gone i'm not going to deal with that right now come back to that then i'm going to say await now there should be a callback that i can give this so that i can sort of console log the loss as it's training over 50 epochs so i could call and and by the way if you're like what is a weight what is async hey boy do i have a set of video tutorials to sell you i have no magic way to make them appear here but i'm sure if you search await async coding train you will find those video tutorials that go over promises and all of this sort of syntax okay um so now um i want to look for the callbacks x y args verbose will it just log things automatically if i make verbose equal true maybe it'll log stuff automatically for me so what i was planning to do is put these callbacks in because basically i can have oh there's a lot of them i can have a call back on train begin on epoch begin and then i can look at the logs which have the loss let's just try running this i turn into a pumpkin in eight minutes hey that worked what now the loss didn't really seem to do anything i mean i am training it off of random noise that was like so fast okay hold on let's make um let me give it up maybe make a thousand random images just to make it uh more data that was way too fast for me to look at and also um i want to artificially slow it down okay great so i have a feeling that i'm not getting anywhere because i'm feeding it random noise but i don't have any errors so i could look at one of its outputs and make it into an image but it would just be noise so i have to assume that i'm doing this correctly on some level um i did kind of one thing that i might like to do is i did in the code specify a learning rate which was not specified in the example so like for what for example what if would this work if i just do this and this will be sort of more default stuff just curious same same results but i just you know there's no reason for me to get really specific about the hyper parameters here when um what is the i'm gonna put this back in this is very exciting to see this work um zarus phil why don't you apply gaussian noise that's a good idea but i'd have to like write a gaussian noise function in javascript although i think i could get one really quick all right that's i like this i have five minutes um i i think i might have implemented the gaussian the random gaussian function in p5 so i think i could just grab that i know there's one in there source math noise no noise would be actually noise math just math no random uh where's the random gaussian so this is the implementation of a random mean okay i think i can i think i can manage this oh but it's not really going to be gaussian the gaussian noise i was sort of thinking of i think this is this is too much effort right now it's not really going to give me i should move on to actually generating the shapes so if you ever wanted to have a reason to tune into the next live stream which will be this coming friday if i had to guess around 10 a.m eastern it will be part two of this auto encoder like i've got all the code with tensorflow.js for an auto encoder right here in node the the only i've spent some time explaining what an auto encoder is i can't emphasize it enough that um you know if you're looking for something to wah uh two minutes like um this particular video will be incredibly helpful it's just like three minutes long so this will give you excellent background to what i'm trying to do to wait uh for next uh for friday when i continue this but what i want what i need to do now is replace this generate image function with some code that actually goes into this data folder and pulls out each one of these images and translates that into an array of 784 numbers then will the loss as i train the auto encoder consistently go down and could i do start to try applying the auto encoder to some applications like denoising or generating new squares what happens if i train it with circles and squares can i morph through the latent space of square squirkles oh there's so much that could be done um so um but i'm gonna just leave this as where we are today uh this is this has been like the most like semi-serious coding lecture uh live stream that i've done in a while curious for your feedback how this compares um um i hope you know we'll see we'll see what the future brings but uh i um i doubt this is i'm gonna hit stop on my record to disk engine let's stop i'm just curious by the way to look at that file i'll try to answer some questions but the file i just recorded today is gate modified 4k obs 11 15. gigabytes ladies and gentlemen and others sorry uh um uh yeah so that probably worked without a lot of uh that's probably a high quality video 57 gigabytes of the entire all my different feeds today if this ever wanted to be edited into a more step-by-step tutorial just from this live stream uh training on purlin noise would be interesting oh and uh i should probably pick another loss function so uh uh k weekman if uh let's chat in the discord um later so maybe you can give me some helpful tips and advice of some things i might want to revisit when i come back on friday let me go back to the chat to see if there's any questions i can answer [Music] before i go is is tf slower in node.js so actually this is a really interesting question um and there's no like easy answers to these questions because there's so much it depends but tensorflow.js in node should be faster than tensorflow.js in the browser because it is uh um it is linked to native tensorflow so tensorflow in the browser is running a tensorflow.js implementation that all the math happens with um webgl context or just the browser cpu tensorflow.js in the browser sorry in node is running actually native tensorflow so i what i hear is that cpu tensorflow.js and node is equival is mostly equivalent to gpu webgl tensorflow in the browser i don't know if that's totally true but obviously the fastest thing i could do would be to run tensorflow.js in node linked to like the the gpu of my computer i don't know if i'd have to install cuda or some other kind of thing for that but i'm not doing that now um yep uh what type of a bunch of other questions are being asked that i don't know the answers to um thanks everybody for tuning in i'm just gonna keep playing this song i will be back on friday to continue this auto encoder project i don't know if it's going to happen but a whole new coding challenge video it's about 30 minutes long a lot of production value stuff went into it which i don't which is who knows that's a good or bad idea but it's coming out this week it is the video from the adopt-a-function p5.js uh series um if you're um if you're if you're on nebula or if you're a coding trained member you will get to see that early so you can join through youtube memberships or sign up for the curiosity stream bundle that i should probably not be talking about because today's video is sponsored today's stream is sponsored by brilliant um if you love the cody train and learning stuff for me i think you'll also love learning from brilliant um sign up for free brilliant.org coding training okay um join the discord say hello there i will see you all next time this coming friday probably in the morning stay tuned follow my twitter join the discord all that we'll get announcements uh subscribe and click the bell i guess that'll do something if you want to just get there's lots of ways you can get notifications about when i'm live streaming but really turn off all your notifications you don't need them from me but if you wanted them you could get them i don't know what to say about that you probably have too many notifications it's really fine you don't need to have another one goodbye everybody as always i always forget that this stop this stop this stop this stop i'm gonna do this stop this stop i'm gonna do this this stock this stop just stopped [Music] [Music] never forget this [Music] never forget this dot somebody composed that song for me [Music] i'm going to say once again here we go [Music] it's look forward to cartesian coordination [Music] [Music] autotune and the internet will fix that for me [Music] song cartesian coordinates [Music] unicorns and rainbows and cupcakes what else is there yes kittens thank you very much kittens and rainbows and cupcakes notice that look what i get i'm really losing my mind okay let's do it [Music] the kittens the kittens the kittens kittens and kittens and kittens the kittens [Music] the kids kids [Music] [Music] [Music] i feel just sort of like a nice feeling of relaxation everything's gonna be okay today dream is not broken it has not frozen this is a this is a wonderful thing okay we're gonna do it i'm really getting to something i need my sound effects [Music] what else is there unicorns and rainbows [Music] generation analysis things that i will use continuously over and over again first thing i need to do is yes [Music] [Music] you

Channel: The Coding Train

Views: 12,494

Keywords:

Id: Y9w2PYfIf34

Length: 128min 55sec (7735 seconds)

Published: Mon Nov 15 2021