Coding Challenge #99: Neural Network Color Predictor

oh hello and welcome to another coding challenge now this coding challenge is number 99 which means the next coding challenge is number 100 and I'm going to go what I feel this pressure to do something special so please in the comments write your suggestions for coding challenge number 100 and maybe I'll think of something or you'll help me think of something ok so what's happening now I am going to do a coding challenge to make a neural network color predictor this is based off of project made by Jabril I would check out his YouTube channel just link in this video's description he has a video called color predictor machine learning demo that I'd encourage you to watch he also actually came on as a guest and I'll link to a video with Jabril where he talks through this color predictor but I'm gonna make my own version of it and I'm gonna use my toy neural network JavaScript library ok so first before I start coding I want to spend some time with you you were just taking deep breaths together think about flowers okay well III guess I should talk through what the problem is so here's here's the problem so to speak now one of the reasons why I love this idea which came from Jabril is that I'm always looking for really simple almost trivial scenarios to demonstrate a machine learning concept where they all the pieces of how the algorithm works it's visual it involves interaction involves drawing because this to me is a good basis for people watching and learning to then build their own more complex or sophisticated design machine learning system things so this is incredibly simple and in fact just to be clear you do not need a neural network for this it almost makes no sense at all to use a neural network for this but it makes the point you might have heard this you might have heard this idea that a neural network is a universal function approximator so I think that this video in this topic in this demonstration will unpack what this means in a in a nice way what do I mean by that so what is the problem that we're trying to solve so the problem is let's say I have a color some RGB color some RGB color and I want to put text on top of that color so I could make a more complex problem which I would encourage you to do as an exercise after watching this what would be the most pleasantly looking or complementary that's a technique that's actual thing color to overlay on that RGB color but I'm just gonna ask a simple question which looks better black or white and we could get an old discussion of why you know perception and what looks better but I just mean just in the sort of arbitrary sense like which is easier to read which is more legible and we could write a function right we could write a function a JavaScript function I'm just checking to see right and that JavaScript function takes as its arguments on our a G and a B and what it does is it returns maybe it returns I mean black or maybe somewhere else in the function it returns white and maybe we just have like this if statement and maybe I'm doing something like if R + G + B is less than 100 or less than 200 returned black otherwise return white so this is the idea this is a function it takes inputs how many inputs 3 and it returns an output how many outputs 1 but out of 2 possibilities is important here there are 2 possible out out - 2 possibilities it takes one output it to function it takes three inputs and returns one output the inputs are numbers between 0 and 255 and the output is one label which is a string but another way I could think about this is it could return a kind of probability value how likely is it that black looks better and how likely is it that white looks better and that would be that would be kind of also 2 so 2 floating point numbers you could think of it that way so this is a function so a lot of machine like imagine this so now this function takes the RGB color now let me tell you let me give you another function right a function that takes a image data of 200 by 200 pixel image and then returns what's in that image now you could imagine in this case deciding whether it should be black or white is just a matter of okay should i if it bright as a dark which color is gonna be like a bright color looks better on a dark color a dark color looks better on a bright color but if I took in a full image I needed to return whether it's a cat a dog a turtle a coffee mug cell phone a walking stick a conductors baton I don't know where my brain is going here a toy tricking right try writing a bunch of if statements to do that well if the pixel colors are this and shaped like this right having to hard code and algorithm a function that takes inputs and generates an output would be really difficult this is what a neural network is for and just to make this case here the idea here is that a neural network can approximate any function it can learn to receive inputs and return the outputs for any input data for any sort of problem in field is this true that at Google any in capital letters the big underline is true that's an open question what are the limitations what should and should not we be doing should we even be using a neural network for this task are we causing harm by doing this machine learning task but those those questions aside now we can say well if I have this quote-unquote neural network thing what if I were to just send three inputs into it our G B and then I want to receive two outputs probability of black probability of white this is now a universal function up approximator it is going to stand in so what is the neural net machine learning doesn't necessarily replace you can think of it as I mean it will machine learning replace the need to write code completely at some point maybe but here what I would make the case is that machine learning a neural network might replace the guts of a function that you might hard code otherwise okay boy am i spending a long time explaining this so now how what goes here what goes here now if you want to learn more about the structure of a neural network and the internals of it I'd refer you to the three blue one brown video series as well as my video series which goes through building this neural network library in JavaScript for us as the user of the neural network library the only things we need to decide are how many how many inputs and how many outputs and then so this is inputs and outputs are the things we as the end user of the neural network look at and control we're sending in the input we're reading the output we're doing something with the output we're sending you the input we're reading the output we're doing something with the output but those sort of quote-unquote magic which isn't magic it's just math numbers multiplied and added together all sorts of other stuff is this idea of a hidden layer and there could be multiple hidden layer there can be hidden layers of different nodes but for the sake of argument this is such an incredibly simple problem who knows if we even need the hidden layer for it we probably do but we can just kind of come pick something somewhat arbitrarily so I'm gonna say there are three inputs there are two outputs and what I'm going to do is just say this is what Jabril is so why not use the same I'm gonna add three hidden nodes and the idea of a neural network is the inputs all go into each hidden node they get processed by the hidden node they each had node connects to every output whoops I do that and then they get processed by the outputs and we get the results that's known as feed-forward what is that processing it has to do with the weights of the connections the summing of the values the activation of the neural network I think at this point it probably makes more sense for me to refer you to my other tutorials that go through the Macan this I just want to now use it and in the sort of higher level way as a library where I'm going to send in my inputs and look at the output and of course I'm gonna have to train it I'm gonna have to teach their neural network to give me certain outputs that I want so that's what I'm gonna get into when I go and write the code right now okay so first let's take a look at gibreel code he might have a newer version by now but this is what he demonstrated last week on the coding train so I could say like oh I think white looks better on this color by the way I'm like have no ability for our talent for visual design whatsoever saw him to get all the track I don't know black looks better black what's better white looks better so you can see this is me active this is this dot represents the computer guessing which ones it thinks it should be and me clicking has to do with me giving this or like training information like hey neural network it should be this one so I'm gonna go through a bill all the pieces of this there's some other stuff going on in here where Jabril is using a genetic algorithm and there's like this sort of voting thing going on but let's let's go let's just start building some code from scratch and we can kind of compare and contrast or you can compare test on your own later ok so this is my color predictor I'm going to go to an empty sketch the first thing that I want to do is I'm just going to create variables for r g and b and when the pro i'm gonna write up maybe i'll write a function pick color and I'm just gonna say R equals random 255 GB and I am going to then draw the background RG B and I'm gonna say mousepressed pick color so let me make a few key points here me in creating this example in writing this code I'm not thinking about interaction design I'm not thinking about visual design I'm not thinking about optimized efficient code I just want to sort of demonstrate the idea and get something up and working quickly you the viewer can then take this and make a more interesting thoughtful designed version of it even perhaps with a different algorithm or a different problem altogether but let's just see now this should be enough code for me to every time I click the mouse get a new random color okay so I might as well also pick that color in setup now I also want to draw let me say text size 64 then I want to say I mean - no stroke I think if text can have an outline in a fill I'm gonna do fill 0 text black and let me do text-align:center also so black and that should be what is my let me make the dimensions of this a little simpler 400 by 300 so this would be 150 then I guess Oh 150 150 I don't know and 250 150 and this would say white so let's see how this goes all right so that's a little bit too big but I could also just make it wider 600 and then this would be 200 and 400 right there we go okay this one should be fill 255 okay so now I have a system where at least I am seeing which one it should be black or white I mean I'm seeing both colors written on top of the background let's draw a line down the middle I think maybe visually it needs that I don't think i centered these correctly but whatever okay okay we're getting somewhere Center center someone in the chat is telling me to use center center why not so that aligned it Center vertically - beautiful thank you well that was an excellent suggestion okay so now what I want to do is I guess what we're ready this is what I love about this problem we're ready for the neural network because we could do it so what do i what do I need I want to make a I'm gonna call it brain I'm gonna make a variable called brain it's gonna be the neural network now I just don't get a neural network in JavaScript just by the nature of programming JavaScript I'm getting it because I have imported already into my HTML file to files and nsj and matrix digest this is a little toy neural network library that I developed in a whole set of video tutorials at some point in the future I'm going to replace this with this new project called tensorflow j/s which is a lower level machine learning library and almost also ml5 is this other library but I'll come back to that another time I'm still using this little toy neural network so what I want to do now in the code is I just want to say brain in setup I want to say brain equals a new neural network now it expects three arguments three arguments now this is not universal to how neural network libraries work this is a very simple one and is very basic features and what it expects is how many inputs how many outputs and how many hidden nodes but not in that order inputs hidden outputs so this we can see is 3 3 2 that's the architecture the model architecture that I have designed three three two wonderful now I've got them done I got it boo boo boo you would know I keep going now what I could do is let's say every time so let's make let's let's let's make a variable called which like which one is better black or white and I will just start with saying black and what I'm gonna do let's do the same sort of technique that gibreel did if which equals black then then I'm gonna draw a circle a circle which is where at two hundred two hundred three hundred sixty sixty and I there and then else if it's white then draw it at four hundred and so this would be fill zero this would be fill two fifty-five and we're still no stroke okay so now we still have to be very far down so let me move this up to 200 that's no sense of dimensions whatsoever perfect oh this looks weird now it's not all centered but fine oh my god I can't I can't take it I can't take it let's let's move this I really shouldn't be doing this but I'm going to let's make this 100 100 and let's make this 200 200 oh okay I feel better now okay so it's always going to it's always going to pick black right now because no matter what's I've just made which equal to black but I can use the neural network now I could use the neural network the neural network is my function approximator let's actually let's let's actually write this code with our own non neural network first just to make this case I'm gonna write a color predictor and I'm gonna say get an RGB and now if I'm gonna just say if R + G + B is greater than 300 then return black else return white so I'm gonna do a hard coded this is my own human learning algorithm I've decided that this is what it means to predict which color would be better and then I'm going to say let which equal color predictor RGB so now we can see it's making that prediction based on my algorithm I wrote an algorithm to make that prediction now so we've got this we could be done know was she learning necessary I finished this project now what I want to do though is I want to chim to comment this out and I'm going to say what am I going to say I'm going to say first I need to make some inputs so the neural network library expects as inputs right my library expects and this is pretty typical of any kind of machine learning based library that you might use it expects the inputs to come in an array of three numbers and typically you're going to want to have those numbers normalized between zero and one so this is what I need to send into the neural network so inputs equals an array and so how do i normalize these values I can just divide them all by 255 and again more likely this is going there's going to be a much longer process of sort of cleaning and normalizing your data but in this case of a single color super easy to do now what I'm going to do is I'm going to ask for the output from the neural network let outputs equal and I'm going to say brain dot and the function write the function to do the feed-forward algorithm to send the data through and get the result back in my library it's called predict because I'm making a prediction another term for this might be inference guess that type of thing so I'm gonna say brain dot predict and I'm gonna pass in the inputs now let me just let me just console.log those outputs just so we can see and this is gonna sort of break but let's just see what the outputs look like whoa hold on why is this to okis draws right so one thing I just realized is I'm kind of I'm doing all this in the draw loop which is sort of silly so let me actually just say no loop and then in pick color in pick color which is where where does the pick color right here I'm gonna say redraw so I only want to like redraw the canvas update I don't have anything animating so I don't need the draw loop to be going over and over again so cannot read property predict of undefined Sketch up yes oh you know what oh why do I have that bug I created the neural network after I call to pick color the neural network needs to exist before I call pick color okay that's good to know great so we can see look at this and why do I have this happen twice twenty nine why is it happening twice you know I guess it's going through the draw loop once so maybe yeah interestingly okay I'm not going to worry about I'm not going to get I'll fit all of that later the point is oops every time I click we can see these this is the output of the neural network it's an array with two floating-point numbers and those I'm considering to be like the probability right if this number is higher it should be a white maybe black is the correct color if this number is higher the other one what's my hand disappeared the white color should be the one of ik now I have to I haven't implemented some things there's a particular algorithm which I really should put into my neural network library called soft max have to make a video tutorial about that in the future which would ensure that these two numbers these add up to a total of one and really represent a probability but my neural network is very simple I can just look at which of these output numbers is bigger so I can say then right here I can say now if outputs index 0 is greater than outputs index 1 will make that mean return black otherwise return white so now I have my color predictor function no longer uses a hard coded algorithm it uses the results of sending the input data through a neural network so let's go ahead and run this and I can click that you can see here it's kind of always picking white by refresh still always picking white by refresh it's kind of always picking black so what's going on here how come this isn't working why is this not learned properly which color should go on top of the other color guess what the the entire mechanic and all of the settings all the parameters all of the weights of all these connections on the neural network were initialized completely randomly a neural network isn't just going to learn as if by magic it needs to be taught and there are lots of different strategies for training and working the neural network one of those strategies is something called supervised learning and you probably can't supervised learning which I have covered in other videos in particular my doodle classification coding challenge so you might look at that as an example but what's going on here so in a sort of normal bit more true data science driven machine learning context we might prepare a giant training set like I'm going to make a big spreadsheet of every RGB color I can think of and which one looks better black or white that's my training data set to pass through and train this neural network with then I'm going to have a testing data set and that testing data set I'm not it's not part of the training data set because I don't I don't want the neural network to know about it but it also has a bunch of labeled data colors with black or white labels then I'm gonna pass that through and see how well the neural network does guessing against those and if it starts to do well then I could say my model is complete it has been trained I can save it and I can deploy it in some application which has to pick black or white on the fly but I'm not gonna do any of that in this video I'm gonna live in sort of a loosey-goosey interactive world where I'm just gonna let it guess randomly and I'm going to click in order to correct it so I'm gonna I'm going to train the neural network one data point at a time with no training data no testing data just random data as I go so you might think about how would you restructure this to in a more sort of traditional training testing deployment context okay so I want to click what I'm gonna do here is as the trainer I'm gonna click on the side of the canvas that I think looks better like I think white looks better so I'm going to click over here just to make this a little bit easier to follow I'm also going in to draw I'm going to draw a stroke wait for stroke 255 0 I guess I'm going to align with / to 0 with width / 2 height there we go so I'm gonna draw this so I can click wise these are totally not centered at all I'm like a lunatic either they're not in the right place i I'm sorry I have to correct that it's making me crazy it is how wide is the window it's 600 wide 300 is the middle Oh silly me 15350 thank you very much No 450 450 thank you very much okay 152 please bear with me 450 okay now we're doing well okay so now the idea here is every time I click over here I want to teach the neural network which one I think it should be so how do I do that so I told you there was a function called predict and the function called predict would send in the input data and give me an output prediction now what I want to do is I want to use a different function called train so each time I click the mouse where is that mouse press before I pick the new color I want to determine is the mouse on the right side or the left side so if so so I want to create some inputs which is an array and we just let put inputs and if Mouse X is greater than width divided by 2 then the correct the correct actually so I'm sorry I want to create some targets this is known as targets I mean there's gonna use different terms for all these things but targets are the target outputs I want so if I click on the right side I want the target outputs for white and white means the second number is greater than the first number so the targets if I click on the right should be 0 1 these is the correct this is the correct output if that I that I'm telling the neural network should be if I'm clicking on the right side else the targets and I know I could use one of those ternary bla bla bla things but this is just going to have do the targets are if if the left should be one come zero let's pick up the pace here people by people I need me not you you're doing you're doing great if you're actually still watching this and now what I'm gonna do is I'm gonna say brain dot train up I need those inputs so the inputs are the same exact thing I did here the inputs are the current RGB and what I want to do is I want to say hey train brain train yourself with these inputs with these targets and in fact this now is going the neural network is going to what's it going to do I'm saying here are the inputs here are the correct outputs that go with those inputs do whatever adjustments you need to do whether you were right or wrong just figure it out and what is that figuring out so interestingly enough I knew this is worth even though this is covered in much more detail in my other videos let's say I let's say the neural network I feed in some inputs and what it actually gives me is like 0.8 0.2 right this is what it gives me as the outputs but ice cave it I'm training it I'm gonna give it targets and the correct targets are 0 1 that's what that's the output that I wanted to get that's the correct output so what the neural network does is actually calculate something called an error and the error is really simple it's simply the desired the desired - the guess or the targets - the prediction or the targets - the outputs so the error would actually be negative zero point eight one - and point zero point eight interestingly enough very symmetrical there so this would be the error and then what happens inside that train function an algorithm called back propagation happens the back propagation takes this error and sends it backwards so when I do prediction I'm sending the data forward through the neural network the training process is about looking at the outputs calculating an error and sending the error backwards through the network and all these little changes all these weights that are adding up numbers new they all get adjusted so the errors would adjust all the parameters and that's what's happening again you could dive into my other tutorials which go through this in more detail but that's what's happening right here in this function so we are ready to go right all right so I'm gonna what I'm gonna do maybe we'll do I'm gonna train this for a while you'll watch if you're watching the edited version this it'll speed through fast if you're watching this live here we go okay I'm back so I tried training this for a while I tried talking about it for a little bit I didn't really get very far even though I think you can see like I I'm gonna like I'm picking and it's kind of actually whoops no I should really move it's giving me sort of different results that's black this one is black that's correct that one is white that's correct look at this yeah hey I think we're good we could see that it's not kind of making some decisions we think this one should be white let's correct it that seems right this is definitely right that seems right this seems right this seems right right right black is better with this one white is better I don't know why anyway you can see it's kind of getting I don't know that I really give it enough training data to really work optimally and maybe I could be an interesting project if this were like deployed in a distributed way on the web and thousands of people could all come and click on it and like through and train it together as a group of people in the world but let's try training it automatically to see if that works a little bit better so how are we gonna do that so let me go back to the code remember this nice little bit of code I had here why not let's train it to actually learn a certain threshold so I'm going to I'm gonna write a function here called train color and it gets an RG and B and it's going to return black or white and what I'm going to do now is in set up the a little bit silly before I do anything I'm just gonna say for let a equal zero is less than 1000 I plus plus and I'm going to pick a I'm going to pick a different set of random colors that are different from the global random colors then I'm going to I'm going to say the answer is what does a train color train color with RGB and you know what I'm gonna do I'm gonna do this targets let's let's skip a step here and this I can't remember which is which whoops let's just return the targets is this right somebody in the chat tell me if I've got these back routes or not let's just return the targets I'm kind of skipping some steps here and not now I'm not being a thoughtful about how I'm organizing this but I want to get those targets then I want to say the inputs are are divided by 255 g g / 255 b / 255 and this should say inputs and then i'm going to say brain train inputs with these targets so this is me just running through and kind of quickly using a thousand colors to train it and i'm being told that this is probably backwards there's no way I could have possibly guessed that correctly okay so let's and actually I'm gonna even I'm just gonna let's let it run ten thousand times so I'm gonna run ten thousand colors through the network right and set up I was like training data basically okay so now let's just and actually in a way what I'm gonna do now just to see I'm not even gonna click I'm going to not bother to train it anymore I'm just going to be I'm just gonna pick let me comment out my own interactive training and let me just pick new colors okay so so every time I click yeah you can see it's making guesses are they good guesses I don't know but they I bet you those guesses are pretty accurately aligned with that threshold of 300 so I could continue to train it a little bit but I feel pretty happy now with this color predictor okay what's what's going on I've finished this coding challenge I'm gonna release this code what can you do with it so a couple things one is could you make this same exact project but instead of having it just pick whether black or white goes on top could you pick a RGB color that looks nice maybe it's a maybe you could have a neural network solve the formula for complementary colors could you think of a more thoughtful way to design this could you use some other data entirely could you train based on like font which font to pick could you train a system to make design decisions based on you some set of training data or some user interaction something like that so I hope you make your own neural network design predictor thingy and share it with me you can go to the Cote trained comm website there's some instructions there for how to contribute your version of a project like this you can write in the comments you can tweet me at Schiffman I guess I'm supposed to say you should subscribe to the channel blah blah blah thank you for tuning in and there is the color predictor wait hold on stop timeout I'm getting a good suggestion in the chat what it might be nice to see like what the number is so hold on let's this is a good idea let's do this hold on hold on timeout everybody where do I have I have some console.log going on here let me get rid of this console.log and let me in here let's console log R + G + B so let's see if this value is bigger and I'm just going to floor it so it looks I don't need to see the decimal let's see if this let's see if it really learned the threshold of 300 right 319 that should be white so it didn't get that right woops 376 ah 4 42 - 89 oh wait no no he did get it right I'm done I've got it backwards if it's if it's higher than 300 it should be over black if it's lower than 300 it should be over white and we can see here right 289 white 431 black 550 13 black 561 black 527 right this is working and just to prove this point let me run it again with a different threshold where do I add that in the training let me give it a silly threshold of 100 that's much too low for it to be visually correct probably whatever that means but we can see now it's going to only go to white if it's below 100 oh it didn't actually 97 it didn't even get there now the quite I'm gonna have to do this for quite a while to get lucky enough to pick something lower than 100 I'm very unlikely to get a number lower than 100 because I'm picking three random numbers between 0 and 255 so I actually have I'd have to pick three random numbers basically lower than 33 and which has a pretty low probability of doing that in a row so let's do the opposite thing let's say let's do this only if it's greater than 500 296 white so if I ever get it greater than 500 black write 393 is white so it is learning that threshold whatever I kind of whatever I in and I pick 300 and probably what would make sense for me to pick is just you know 256 or 20 55 times 3 / yeah - or 55 times 3i / - I really should have stopped this video of two or three minutes ago when I had the chance so we can see now here's my color predictor it's trying to predict which text looks better over the colorbond season now we're finally done oh please please creative wonderful people of the internet make a more interesting better version of this and I will see you in a future coding challenge good bye thank you again - gibreel for this particular idea subscribe to his channel link in this video's description [Music] you

Channel: The Coding Train

Views: 75,674

Keywords: live, programming, daniel shiffman, creative coding, tutorial, coding, challenges, coding train, the coding train, live stream, itp nyu, class, challenge, codingtrain, code, neural network artist, artificial intelligence, neural network, color predictor, jabril, neural network library

Id: KtPpoMThKUs

Length: 37min 45sec (2265 seconds)

Published: Tue Apr 10 2018