Seriously, let's *really* finish the AutoEncoders Project!

[Music] do [Music] hello [Music] [Applause] [Music] [Applause] [Music] do [Music] [Applause] [Music] [Applause] [Music] do [Music] do [Music] do [Music] hello good morning or afternoon good evening a little bit of a sound check here quick sound check um there's probably a loud pun because i am currently running the heater in the garage but other than that let me know how my voice is sounding and i'll be getting started in approximately two and a half minutes [Music] [Music] do [Music] do [Music] me [Music] do [Music] do [Music] hello happy monday welcome to another session of the coding train my name is dan i will be your 2 conductor for today's session i'm still setting up a few things here and there last time i forgot to record my session to disc uh so i am going to make sure that i do that this time and i want you to think about today's sponsor of the coding train as i go and set those settings today's sponsor is curiosity stream uh let's see how come that did not open obs 64. oh obs is already running launch anyway yes okay thank you and now i am going to hit um start recording and there we go all right so um [Music] boy boy am i unsure about what's happening in the world of the coding train but i am preparing i don't know where it is maybe this is it this here is a stool a stool actually it's like one of these weird stools that like you sit on and it wobbles it's supposed to be good for your back but i am now hereby proclaiming that this is a reset button it's a giant reset button and very soon it will be 2022 and i am going to place my bell on top of the reset button and i can't do it now the ball will drop drop onto the bell we'll make that sound uh and i'm gonna everything's gonna be reset um in the year uh 2022 because this boy um and um this has really been quite a difficult uh i know year two years for so many people just for me speaking personally just a really difficult month of december and i haven't been able to sort of keep up with the pace of sort of coding train activities like i would really like to but i'm finishing up a whole bunch of things and have this clear path ahead of me so this is the time by the way if you ever wanted to get in touch with like your suggestion of what you think the coding train should really be should it really just be daily live streams every day should it be more sequenced video tutorials should it be project videos should it have high production value low production value uh should should it have a brand new website that allows you to share all the things you're making based on the coding trade videos back should we be doing more things on social media should coding train have a tick tock i'm thinking about all these things and um ramping up my plans to uh to activate uh all of the all of the cars on the train and the various different engines and the components it's all gonna happen in 2022 but here i am still in 2021 just eeking out one more live stream there's gonna be another one at some point i really would like to do my annual holiday slash new year's processing foundation telethon to support the work of the processing foundation i think this is this is not official yet this is unofficial this is like a little unofficial pre-announcement announcement i think it's going to happen after the christmas holiday but before new year's um this year i don't think i can get it together to do it this week or next so most likely my uh new the holiday livestream which will be a new year new year live stream launching the new year of the coding train will happen it's probably i'm gonna guess uh december 29th or 30th or 31st one of those dates if you want to be the first to know when i've scheduled it you should sign up for the coding train discord i just pressed a button and i don't know if that message is going to go into the chat didn't seem to do it but it's uh codytrain.comcodingtrain.comdiscord discord.gg codytrain somebody will put it in the chat hi peter glad that you're here catching the stream um but uh what am i talking about if you want to be the first to know about when this special end-of-year new year spectacular coding train telethon spectacular will happen maybe i'll even break out my ukulele to sing a new year song would you like to write that song the lyrics of that song get in touch because i got nothing i got nothing if you want to be the first to know sign up for the discord because the discord oh it has this new feature let's see if i can even show it to you um i'm going to open up discord here on my computing machine that you cannot see just yet oh no okay wait wait wait this is fine i have to log in everything's going to be fine it's got this magic qr code thing so this is really what i should be doing on a live stream i should be logging into things start talking um i'm talking already i don't need to start talking discord why you're reminding me if i had a co-host i wouldn't have to just continuously speak the entire time i should be good with the silence or i need more music going on um now let me just do this little special trick thing that i'm going to do then let me go back to ah the coding train oh i'm really doing something very important but you can't see it because otherwise it will reveal all of my coding train oh my god secrets um um we wouldn't want to do that um now okay yes yes yes no no no i got one more button to press boy i really should have prepared this did i tell you about today's sponsor look at that well i'm still doing this hidden option command i uh turn this off oh boy this was boy this was worth the wait everybody oh you have no idea what's coming to you you have no idea the excitement i'm about to show you such a letdown but you'll see here in the coding train discord now has this events feature so whenever i have a live stream event i schedule it in the discord which there's a way for you to get a notification about that you can see the youtube link the details about it multiple it's if it's happening now it have a big happening now button so i fully expect that there are now hundreds of thousands of viewers in real time the kodi trade right now because of this new happening now button thing card tag discord extravaganza all right so um you can also just subscribe to the youtube channel and there's like a bell or something it'll also give you a notification in theory when i schedule the next live stream uh and simon is reminding me um in the member discord remember oh remember when you did advent of code on the stream yeah december is i really want december to be like a really active month of streaming and coding and fun but it's just not happening this year but mark my words streaming and coding and fun and all of that stuff is coming in next year so what are we here to do today if you've enjoyed my last three live streams do i have good news for you i'm gonna be continuing the project this is kind of a new thing that i started which is i'm just going to come back over here for a second which is to work on a larger project over multiple sessions i don't know how effective or entertaining or insightful this is so right now the kinds of things i do in the coding train are divided into sort of different different buckets and actually one thing i might just do right now just to talk about that for a second is to come here to the channel so this is great for anybody who's who's new uh today so i have what the most popular thing that i do which youtube likes to tell me is very popular are these coding challenge videos from at least four or five years ago so apparently the way that i work is i discovered something really popular that people love and want to get more of and then kind of went did something different but so uh these are the coding challenge videos they're sort of standalone one-off videos where i build a project i also have and this is kind of where what i'm kind of very much dedicated to always having at least this feels very primary and fundamental to me um hi christine in leeds england welcome to leeds england lovely to see you here um these are the sort of sequence tutorial videos so first of all if you are new or if you have a friend who's like oh i'd love to learn to code or where should i start or i'm interested in generative art or creative coding or javascript this playlist is for total beginners and one of the things that i have been doing and the reason why i like having it sectioned into different videos is because i can over time replace them so if i look at this right now these were um you can see like these are from three years ago the sort of intro to the series and using the web editor then all of a sudden oh yeah then we still got three years ago then all of a sudden we go and we get some videos from six months ago more gray hair so these were so out of date and old that i started to replace them new thumbnail style i don't know what people think of that um but then we go back and we continue like suddenly then the video goes to six years ago so definitely high on my list is anything that's five plus years old in this sort of tutorial series i want to redo i have been working this is what i'm kind of focusing on today i would like to make a video about auto encoders uh matt is saying i could probably get a co-host in songwriting help from gpt-3 great point actually that's been my sort of pet procrastination project is playing around with a lot of the new large language models that are available through open ais gpt-3 and also hugging face uh hosts a tremendous amount of different language models with lots of exciting applications although i think it's important to be very cautious and conscientious when using these large language models so there's a lot to say about that um nature of code is a big project for me in the new year i don't know where i'm scrolling so oh so so but what i was talking about which i wanted to just sort of get off my chest here is that standalone project videos like these sequence tutorials and then if i go to live streams i thought there was a place where i could just see like my live stream playlist i'm not seeing it i'm not going to to try to find that now but that's what you're watching right now apparently 166 of you lucky lucky lucky people are here with me um and uh would you redo the game you started says nico writes would you redo the game you started with the triangles moving around the map on their own i'm not sure what the pot redo them but never remove them yes so this is a good point i am i have no plans to ever delete any of the older videos and in fact um another way to browse the videos is through the coding train website and if i go to beginners if you really enjoy the sort of classic coding train style uh you can find these really ancient uh intro to processing tutorials which are even though it says 2015 i'm pretty sure these were recorded in 2012. it's just that i only got around to uploading them to youtube all at once on july 10 2015. ah time flies i'm getting older too what's that song that i'm thinking of you're getting older i'm getting older too i don't know i'm getting older you're not you're you look lovely you look beautiful you look younger than you ever have and actually the irony here is while i'm talking to you the viewer i'm just staring at an image of myself so i'm saying it to me but really i'm saying it to you all right we got to get we got to get moving here i've been alluding to it i want to tell you about the best deal in streaming available today oh anything think of any streaming service with any content i have to tell you about the best deal available better than all of the ones you're thinking of um so how do i get to it if i go to youtube.com the coding train and the community tab i just made a post about it and here it is holidays start early so um you've heard me talk about this before um it is the curiosity stream and nebula bundle it gives you access to all of curiosity's stream tons of educational documentaries nature science math um just nature nature nature nature those are the ones that i love so many amazing documentaries and with it you also get full access to nebula which is a streaming service that my content is on you can even see here this picture so all ad free if there's any all sponsor free um really nice player you've got like a roku app and a fire tv app some new stuff um i think i wrote about it here um picture and picture on ios and like i stole this from renee ritchie who is i'm a big fan of all for less than the cost of a usb dongle so um you can get all of the nebula content all of curiosity stream thousands of documentaries for less than 12 a year this is exclusive only for the holidays it's a 42 discount 11.59 uh if you sign up through the link it's pinned in the chat um curiositystream.com codingtrain so if you're wondering like what's a way for you to support the coding train actually going to that link and signing up will uh supports me and supports all these other wonderful creators that are on nebula i'll come back in the middle and just basically say what i just said now again but if you have any requests to poke around and look at anybody's any particular content i can make some recommendations for you all right now coming back i am really excited about this project uh because it is one of these things like uh as simon likes to tell me um wait i'm sorry i'm reading i'm reading the the chats in multiple places uh thank you viveshop vivesvan um simon likes to remind me um i have this habit if i go to the rainbow topics github repository i have this habit of making these like to-do lists fall 2020 spring 2021 summer 2021. notice i didn't make one for fall 2021 i was like forget it i just make the to-do list don't get to anything that's on it i think i would really like to make one for 20 20 20 20 20 20 22 and uh really develop a schedule oh something that i'm thinking about tell me what you think about this here's an idea for you uh think about like some cool illustration graphic design twitch tuesdays coding train twitch tuesdays like a little music bump or something um i think you've experimented with streaming on twitch but that's so cheap so a little different vibe try some more interactive features something i'm thinking about um but i believe like if i just go to summer 2021 you can see all these lists of things that i want to do and on the list it's been there so many times auto encoder so vivisvan asks what are you building today auto encoder except i'm not really building it today i am finishing it today i have been building it you have guess what you have about six to seven hours of old content you can go back and watch so i have been working on this auto encoder uh with tensorflow.js and p5.js uh for the last three live streams i don't think i should go past this one and the thing that i'm thinking about doing and i would love to hear from you is then um not on the level of say like two minute papers or some other like three blue one brown is another channel that i love sebastian lag um jordan herron i'm just naming youtubers that i like but um but i am thinking about what would it mean for me to do a scripted i know shocker scripted video about what is an auto encoder and how to build one using tensorflow.js and p5.js and cut and paste that's not the right term for edit together highlights from the live stream sessions with narration and some additional demonstration and that could be maybe in 20 minutes that's kind of a goal that i have the only video that i've done anything remotely close to this is the mouse learning um let's see if this comes up um this this video if you haven't watched it um let me just pull this up for a second um this is a video that i made that had a script and um i love you too do do do venom okay you just made me say doo doo venom that was a trick good job uh and andrew's asked me i've looked at advent of call so uh if you haven't watched this video i would love for you to go check it out and tell me what you like or didn't like about it but the idea here is could i um could i could i pull could i could i kind of have the best of both worlds like one of my mantras one of the things i really try to do is show every every every piece of building a project start to finish and all of the pain and bugs and mess in between at the same time how many of you realistically if you're like i want to learn how to build an auto encoder oh there's seven hours of video i could watch where at the end you get a fuzzy circle um really have the time for that or going you know is that really going to be uh useful for you so i want to try to have the best of both worlds that's something that i'm thinking of planning to do matt says great plan matt gorbay gorge from the french gold no maybe sorry if i'm butchering your name um so um that i am thinking about um um doing so anyway new area of exploration look at all this stuff this this so much stuff that i want to do oh remember how i was going to do these i actually i have some like just videos that i sh recorded and they're kind of like half edited that i never got together to finish uh so much oh plotter i want to buy a plotter so many i want to get a knitting machine so much to do in 2022. i'm a poet and i didn't even know it all right so let's get back to the task at hand before i take up too much time with this introduction to today's session thank you for everybody too for being here let me go to terminal i'm going to walk you through now all the pieces of the project that have already been built so first piece is a processing sketch to uh generate training images for the auto encoder and i realized that if you're totally new and haven't tuned into the last three live streams the question what is an auto encoder might be on your mind i'll come to that at some point i'm sure i won't be able to help myself but just imagine at least for the one sentence version i'm using an auto encoder to create images in the style of an existing data set and this here if i run this sketch this is now generating that existing data set it is just a series of random circles or squares um um i believe i'm making some number of them as we will find out i'm making 1100 of them so at some point it will stop and we can see now in this data folder here are all these images that i generated so i'm hoping by the end of today i can try to work with something more sophisticated should i try to use letter forms color different kinds of shapes i'm not sure yet more abstract patterns again i don't know um but let's leave that oh fractal trees we could try that would be kind of interesting learn blender so sake yeah i kind of into this idea of learning unreal engine and then turning this garage into a like a virtual set i have this huge garage now can i make it into a volume like the mandalorian set and just put like screens over everything instead of a green screen that just actually be this giant screen behind me now i don't have the budget for that i'm going to need a lot of you to sign up for that curiosity stream bundle if i'm going to do that ah and andrea asks are you working on a new version of the nature of code um and stig writes um oh just crack the playback speed to 28x i hope by the way all of you just watch me on 2x i mean obviously you can't right now well maybe good no touch designer i don't think is for me i appreciate the suggestion from rodrigo but probably not for me um wait wait wait i was talking about something generating images we're doing the auto encoder let's talk about the mandalorian set ah nature of code it was andrea's question uh yes so i you know i don't know why i'm dancing around this i am on sabbatical starting it's actually not the you know i still have a lot of admin and other uh nyu related responsibilities that i'll be continuing to do for the first three weeks of january but my sabbatical from my teaching job begins january 24th um i don't i'm not suddenly going to be full time coding train but if i'm at best you know one fifth time right now one eighth time right now coding trade honestly this month um i'm hoping to ramp up to like a quarter time a half time like really spend two or three two to two like at least two full days per week working on coding train um i also i've got a um there's an expression that uses naughty language if you will that i'm not going to say i have to like do something or else get off the other thing i got to do this nature of code book like it's now and forget it like if i don't have a new version of the nature of code book done in 2022 forget it it's never gonna happen you heard it here first mark my words sign me onto the piece of paper everybody record this broadcast it put it out into the universe i'm either going to have a new version of the nature of code book completed that means you can buy it a print version and it will be all online for free as a website both of those things by the end of 2022 all with p5.js a new chapter on neural networks if that's not done that's it that project is dead i don't think i'm going to do a kickstarter for it which is what i did for the first book i think i've got enough kind of momentum with the sort of support but so those of you who are supporting the coding train whether it is signing up for a membership supporting the sponsors um all the other kinds of ways just watching tweeting sharing recommending it to your friends all the things that all of you do that i so appreciate um i think that's enough to kind of keep me going in terms of nature of code book um yeah uh [Music] unity and godot would be better alternatives to unreal says uh simon okay i'm gonna take that advice very seriously um and thank you kathy wrote this really uh oh nicole shader expert nicole is in the chat oh shaders i actually really would like i don't know if i'm qualified to do this i got to see uh the great pattucio author of the book of shaders i'm in new york city recently and i said oh thank you thank you thank you thank you thank you thank you i feel like that's what i people sometimes like have that reaction to me and i'm like no no no no please no stop oh no no oh if you insist and then i get out my ukulele and play them a song no okay um christmas gift yes okay um nicole uh thank you for your help when i tried i stumbled through learning shaders thank you of course to eliza and aletheia in the uh curiously minded stream i was thinking i might like to do a little basic introduction to shaders as like similar to my intro to p5.js but i don't know i i to do that i really have to think about how i'm kind of crediting um all of the sources that i'm sort of learned from i mean this is something that i do throughout throughout everything but also like whether i'm really the right person for that i mean in some ways i lo that's the idea of the coding train like i'm a beginner at this too so let's learn together i have to think about how to do that effectively okay um [Music] yeah uh nicole you are an expert relat in my eyes but no one expert is uh just a word that maybe we should just remove from the vocabulary here i mean it's it's nice to be an expert in something but i feel like in the so i don't there's nothing wrong with being an expert people who are experts should be celebrated for their expertise especially in like the sciences and things that we sort of like rely on to keep our world turning and our society moving forward but um in the in creative coding i like to think of all of us as um expert amateurs or amateur experts i'm not no i think it's more like we're expert at being an amateur like that's what i am ah i've decided i am but i'm not i'm not saying i'm good at it but like the thing that i like to do is try to learn the new thing and then help others learn it as well and sometimes i fail sometimes i succeed who knows back to the autoencoder project kendri i'm not prepared for this kendri welcome to your coding train membership [Music] you have just boarded the coding train passengers manifest please make sure you sign up for discord check the community tab for a post where there's a google form that you can fill out make sure you link your youtube discord accounts and you also get a random number 2022 2022 is going to be the year that i'm going to get back to reading this book of random numbers in a logical and organized way but your random number kendri is on page 169 it is row 8420 column one two three four and the number sequence is five six zero two nine again um we've kind of lost a little bit of momentum on this but we'll get it back a bunch of you if you're in the chat you received one of these please let give a shout out but um members uh uh at certain levels will receive this beautiful custom laser etched train whistle with a random number with a random walk according to the sequence of your random number of the random number book oh that's a mouthful okay um [Music] ah so um kd kydz i got to get to the auto encoder um it's 11 35 so i'm i'm going to start that but i do want to address this is an excellent question from kydz how do you see the use of your projects on on websites either following the internet video or changing up code i love the flock animation are you okay with crediting great question so let me rephrase the question let me just create a scenario you're a viewer of the coding train you saw one of the coding challenges maybe uh and i'm gonna just go to them and you're like oh this self-avoiding random walk this pattern is perfect for this design that i've uh that i've been hired to do for a movie poster or for my own personal portfolio website i want it to be running in the background um can i use it so the answer the short answer that question is yes with no other caveats all of my uh example code is released under a very permissive license typically the mit license hopefully um it's here in the website i think it's in the faq uh license here we go mit license so this is a short and simple permissive license conditions only requiring preservation of copyright and license notices license works modifications larger workspaces from under different terms without source code so basically you can you can um use it for anything you want you can modify you can distribute it it can be private you don't have to you don't have to provide credit um so there's not a lot of restrictions in terms of the example code um let me say a few things about that um it's nice if you can give credit i appreciate it i'm saying this less for me i get plenty of credit i don't need any more credit for stuff but we live in this sort of ecosystem of creative coding open source everybody's got different sort of comfort level with things being reused not reused so always check to see if there's a license regardless of what the license is i would provide credit a reference a thank you i would be overly generous i would go above and beyond whatever is required by the license and the thing this doesn't apply to my code examples but you will often find artistic works that are released under an open source license so in theory there's no legal issue with you taking that code and using it say on your own website but i do think there is something different to copying example code and remixing it for your own creative vision and taking somebody else's artistic intent and applying that um to you within your own work without sort of proper reference and credit so it's muddy it's murky um certainly there are sort of like clear scenarios like oh look example code under mit license please go forth and use artistic work without the source code being open source you may not like copy that put it out on your own name sell it you know make prints of it sell it on your local corner store market or whatever kind of digital currency things people are doing these days i don't know what i got to talk about right now so um and i think actually um golan levin has a really excellent guide to this i'm just want to like i know where i can find this so i'm just going to pull it up i'm very actually i think i can probably show you this pretty sure that i keep a note to it in my syllab syllabi i know i do for my my undergrad course that is in nyu's learning management system which i can't pull up but i can go to my public syllabi for my uh programming from a to a z course by the way if you're looking for some of my most recent stuff um pablo is saying open processing now requires to give credit to all you yes so open processing is definitely a site where people are publishing both their code and their sort of creative artistic vision and so if you're helped by the code and you provide credit i think that's very reasonable if you are just you know if you took something on open processing and like turned it in as your homework assignment for my class i think that would there be an issue even if that's like not something that's you know that you know you could be sued for you know according to whatever the laws of wherever you are let me just go here to a to z let's see if i can find the reference to golan levin's um yeah here we go so i adapted this um it's called um a statement use of free and open source code from examples uh this is adapted from i'm sure hopefully goal 11 may have an up more up-to-date one from when i adapted it from fall 2018 but i just thought this um this is a really useful uh section um that is is is uh speaking of giving credit adapted from golan levinson written by golan levin be careful tonight happens that an artist places the entire source code for their sketch or artwork online as a resource from which others can learn assignments given in new media arts courses are often similar you may also discover the work of a student in some other class or school with posted code for a project which responds to a similar assignment or even the assignment for your class that you're taking right now you should probably avoid this code or at the very least be careful about approaching such code for possible reuse if it is necessary to do so it is best to extract the components that solve a specific technical problem giving credit there i would add and um rather than those parts which operates to create a poetic experience again these are tricky things to define in really strict boundaries uh with really exact right or wrong we're all trying our best it's very hard i think transparency and narrating and documenting your process is the most important thing you can do so even if you did something that you felt like yeah make your make it your own even if you did something that you know could be characterized as repurposing the poetic aspects of somebody's project into your own if you've documented through your blog post through code comments through your website your entire process of how and why you did that then at least i think a um a dis good faith discussion around that can happen and can be corrected if their mistakes were made so um so yeah so you can re you know you should you can uh hopefully i don't know if this let's see if this link still even works yeah so this is the original um policies from golan's uh celeb syllabus i assume there's a more recent one in 2018 you can find it under my programming from a to z and other courses if that's helpful to you if you have suggestions about a phrase and write that better boy would i love those because this is a very uh it's like you know talk about faqs for people who are new to teaching people are students new to coding uh i ask this question all the time and get confused myself so okay now auto encoder we're really gonna get this project this is why it takes by the way this is why it takes seven seven hundred three hour live streams because i spend 45 minutes i haven't even started working on the code for this project how about we start now i did i started demonstrating okay generated the training images the next step um and let's check i have a node a piece of node code that runs through a series of steps basically it builds the architecture of this machine learning autoencoder model um and sorry i'm just reading k y d z zs i'm gonna this balancing act of like just having this discussion with the chat going through this code example is quite difficult but um recreating the flock type animation um is yeah and and so again to be clear i am making and releasing my examples for you to use use them and you're not required to give credit which is different than if i had some sort of like artistic project that i was displaying that had some other kinds of intentions behind it for you and how you might reuse that so copying my code exactly and putting it on your website is fine that's what it's there for of course you probably want to make it your own that's the exciting part of all this okay um sgl let me see if hopefully somebody in the chat can post that link otherwise tweet at me or follow up after i can try to put it in the video description later okay so back to the auto encoder the code that i've written so far basically does everything it creates a loads all those 1100 training images it trains the auto encoder with the training images i'm using just the first 1000 images to train the model it then saves the model to a to a local file because i'm going to load that model from a p5 sketch and then it also generates test images just to sort of see um okay so um let's run that let's first look at the directory and um i'm going to delete all of the the output images that i generated last time to make sure this is actually working and i'm going to run the training now it was suggested to me i think over social media after last live stream that i absolutely need a song to play or sing by the way i do have my ukulele here in the studio it's been over a year since you know where's the joy in my life that it's been over well over a year since i actually even unzipped this case i mean there's no way this is going to be in tune even this is the night this is my nice one has a strap it's not in tune at all none of the chords i play will work but we need a training song okay i'm gonna have to deal with this later um but so i'm uh anybody who wants to write me a training song maybe it would go [Music] epoch 34 out of 100 eta 1.1 1771 milliseconds so this is not good this is not good this is not good uh the loss is going down it's at .06 this is very embarrassing the voice in the back of my head always when i start to like um the silliness starts to take over a little bit is um people who comment i would have enjoyed your video if you just would stop it with the clown act i can't help it that's my that's my inner that's my inner inner soul is a clown it just wants to break free i never made it to the shock le cox school of mime in paris that was like my childhood dream and i just never made it there and now i reduce to machine learning javascript examples while clowning around [Music] like getting 100 epochs was definitely too long yeah chris sears um it should be in the style of a rocky montage i like um i really don't need to let this go all the way but um especially because i already trained a model but it's it's almost there so i can let this go um uh all right so we're almost there to getting this to train and as i start to iterate and work on this more um i'm going to not you know take the full five minutes or whatever this has been for it to train but we're almost there so once the model is trained um manus asks are you plotting the law so questions would have been good so i am not the loss the loss value which is the result of a loss function is sort of summarizing the overall error of the model how well is it at reproducing those original training images and you can see that it's going down over time although it kind of went up here so one thing that i could do is analyze like when did it really stop like clearly at epoch 34 it was quite higher than at epoch 54. but you can see somewhere around here actually maybe not even until like the 90 around epoch 90 or so um did it sort of stop really going down um kathy asked a great question it is possible to set an acceptable loss or maybe this is a question is it possible to set an acceptable loss before running a stop when it reaches the desired loss that's uh absolutely i could write that into the code i'm not going to do that right now because that would be sort of an additional thing to engineer but i would certainly encourage anyone watching to try to do that and that seems like a really interesting idea yeah so again just to emphasize um hi michael k ah i posted something on twitter and somebody came to join the live stream look at this amazing um so okay so many interesting questions here so let's let me address these questions by the following so first of all this is the output that i have now generated let's take a look at these images and i'm going to just scroll through a bunch of them so you can see i'm getting what i was hoping for which is these kind of like squirkles they're sort of they're mostly circular but you can see the sort of corners of squares and sometimes i got a full square as soon as i have a full circle sometimes i have something somewhere in between now a couple things to mention about this one this is an incredibly trivial example right what i am reproducing with the auto encoder are very low resolution simple shapes that are either only i mean there's really only two variables here there's a switch is it a square or a circle and then there's the radius so you can think about i mean this actually came came to me i mean i think it was based on some of the chat messages but if i go over here to the whiteboard if i were to like come over here and sort of re-explain what an auto encoder is and again uh what is autocoder go and check out the two-minute papers autoencoder video on youtube uh where it describes an autoencoder as a copying machine right it's a neural network that takes an image in as an input and the idea is we want to get that same image out as an output that's a very easy thing to do right we could just cut in terms of without a neural network i just iterate over the pixels and make a new image with all the same pixels however the idea of an auto encoder is the data is being passed layer upon layer upon layer through a neural network being compressed down to from however many pixels to some other layer with a smaller number of nodes and then decompressed back up so if you think about it i technically only need two nodes in this sort of center layer for what i'm doing because one variable right i need to keep track of what is the size of this shape and the other variable is it a square or a circle so in theory if i go to the architecture of this auto encoder and i look at how i've built it right oh i'm sorry i didn't i didn't switch over if i look at how i built it the first layer receives um it has 256 it receives the number of pixels of the actual image and it compresses it down i mean it's not this isn't really compression i'm just using that as kind of a in a sort of metaphorical way to describe the process although i mean it is compressing it essentially the data and 256 to 128 and then down to eight and then back up to 128 back up to 256 and then eventually back to a full pixel image um right and we could do a convolutional autoencoder and a variational auto code and then a thank you astro penguin for answering about overfitting um so um so let's we can address that in a bit i feel like i was getting some good suggestions about changing what loss function i'm using but just out of curiosity i hate to do this let's add one more layer and bring it down to two units and then have the decoder also have one more layer with eight units then let's see if i can still get it to work and i know i said i wasn't going to do this again with a hundred epochs but i can't help it so here we go we are training the model again let's see if that loss can go down and see if we can get um a mean squared error loss function for auto encoders yeah um and astral let me just read astro penguin's comment overfitting is when your model doesn't just represent your data but also your noise it is so well trained to your data that works very well on it but poorly on another data set right you can train a model so well that it can re-produce the training data perfectly but because it's just so locked into that training data if the real world data has any kind of variability to it that's just not there in the training data it's going to explode and do a terrible job so there are lots of techniques to reduce overfitting in terms of how you collect your data set using something called dropout which is adding some sort of randomness and disconnecting parts of the neural network as it's learning um this seems to have settled on a loss of 0.23 i'm going to just run this again and just give it 30 epochs so we can i think that's faster than waiting for it to get to 100. uh where did i where do i set the number of epochs um train here it is so let's just let's just train it with 30 um and let that go um ah yeah um and uh simon's reminding me that i was gonna try to do this with um try to see if i could de-noise some images so i'm i'm getting like sort of scattered here of all the different things i want to try but it's okay it's not even noon yet i've only spent an hour um place the auto encoder song cyber said this is great cyber gus says it's like preparing for a math exam by memorizing the solutions to the problems right so like if you have like a sample test you just memorize all the answers but don't look at how any of the problems work and then you got a new test you could put all those correct answers in but they won't match the actual problems it's kind of a wonderful okay so i think we're at epoch 30. i'm just curious to see oh there's a lot of randomness in how well it does i didn't let you let it go more i didn't realize but let's just see what kind of images we got even from that from that training so they're they look pretty good they're just much fuzzier so this is what it was able to do with just two parameters but also i didn't let it train for as long as i could have nicole wants to know what is in my mug first of all love this mug fellow mug not a sponsor you never know though um it is coffee and oat milk delicious sometimes it is ginger tea depending on how i'm feeling i'm either like a coffee and oat milk or i'm a plain ginger tea kind of fellow that's in my felt with my fellow mug that's me all right moving along um let's go back to um yeah i'm reading simon's commentary and i totally agree let's look at whether or not our auto encoder can effectively denoise a shape this is you know an actual application of auto encoders what do i even mean by denoise hold on so i'm going to put this back to 100 epochs and i'm going to go back and get rid of this like little layer uh the and just leave the the middle layer um as having um eight units um i want to just try it with a different loss function um is it this is this what i should put in here let's just try a little a little risky here to introduce that at the moment but okay let's train i'm just gonna let it train oh look at that huh look at that it like with that loss function it's like not improving after just a few epochs interesting so hold on this will be great if that was that much of an improvement let's have it go just 10 epochs sometimes it's totally unnecessary but i just like to delete the output just to know that i'm getting new output for sure let's run this for 10 epochs and see what we get look at that like basically after two four five oh no it's still improving huh oh it's still it's always why does this happen to me it's still improving so the other time it kind of like was waffling but let's take a look at what i've got after just 10 epochs all right fine torture me so go back to 50. this is one of the problems with this project is just running again takes so long uh mse did i not type the right thing in let's let it go we're gonna let it go for let's let's actually look in the tensorflow documentation um [Music] where is that is this tensorflow.js i don't think so um these are all the tabs i was looking at yeah look at this going way down still going down whoops um this is what i want api um i want the tensorflow.js api uh mean mean squared yeah no i think i got it right mean squared error now do i want the stochastic gradient descent the learning rate was too high um [Music] might be getting stuck in a local minimum the opposite of a t all right um so that's interesting i wonder if i should be also trying this to cast sgd um optimizer instead of atom um but anyway it was still the loss even uh even at that 50th epoch was still going down pretty consistently although it did pop back up there so let's take a look now at the images we generated all right these look really good so what i want to see now if i can get it to do is denoise an image but i'm going to go into the browser to do that that's what we didn't get to do last time okay everybody um okay so the next step of this was um if i run a local web server i have a p5.js sketch which is loading the model that's interesting i i thought i would have to go into like the public oh it's serving it knows to serve dot public that's so funny because i have a directory in there okay so this is now the p5.js sketch which is loading that model and i am feeding it just random noise like a a sort of purlin-esque no perlin noise there's always sort of discussion around whether this is true perlin noise or not or simplex or gradient or whatever but this sort of cloudy noise pattern is what i'm feeding into the model and i'm getting this shape out meaning the model is working and this is kind of like leading the way to what i want to really focus on today which is a latent space walk but before we get to that what i would like to do is draw a um an image of a circle and see how it copies it an image of a square and then add noise and see what happens bear with me or join me for this part so i'm going to go into i'm going to go into the um sorry sketch.js i lost my train of thought here and um [Music] let me just do uh training new model so i am going to i know sorry comment out i think i'm just going to take out the noise thing because we don't really need that so it'll be in the code history do i want to add a tag people going to want to find this well i know where it is so we'll finally i'm going to take out all of the noise stuff and just go back to a random image and saying no loop so if i do this um and then next image sorry i'm forgetting how i'm doing this uh does this work and then put no loop in i just want to know this is silly i don't need to add the no loop but i'm just curious too all right forget it keep the no loop have it be a random image wait sketch sorry hold on what did i do oh i lost this ah okay this is what i want to do so this is what it looks like when i'm feeding random noise into it it's trying to find and and month nonsense we see a machine trying to find shapes in the random noise so what i would like to do instead is and i'm going to create a um i know what to call this the um the i'll call it input canvas and let's say input canvas equals create graphics at um i've done this in sort of like a weird backwards way but bear with me um what is the size oh yeah create graphics w comma w no no we're fine create graphics i have i forgot i had that w value so i'm working with 56 by 56 pixel images um and the input canvas i'm going to say input canvas dot background i'm going to draw a background of white then i'm going to draw i'm going to say a stroke of zero a stroke weight and this i should essentially be matching what i did in processing so i'm going to say a stroke weight of 16 16 then i'm going to say um input canvas circle uh w divided by 2 w divided by 2 and just give it um a a radius or diameter of w divided by 3 and i'm going to say input canvas no fill then um i'm still this is still going to be random i'm just doing this one the only thing i'm trying to do right now is i want to see a drawn shape drawn with p5 on this left part so now i should be able to in draw instead of rendering this input image it should be able to say image input canvas 0 0 and resize it up to w times w wait why won't it why won't it do that doesn't my variable names are terrible where was okay and then let's see if this now gives me oh this always happens it's trying to import ah it's trying to import something i really have to work on my vs code settings okay good oh the stroke oh i forgot i was drawing it as a larger image than resizing it down so this stroke weight was you know a bit much uh technically it should be um uh 16 where to this is very silly but i'm just going to do this okay great so now the idea here is that the auto encoder should take this input circle and produce an exact copy of it on the right now however what's important to note is i'm not feeding it into the auto encoder so the next thing i want to do is feed the image into the auto encoder so [Music] that happens here so i'm going to need to do it by looking at the pixels so i need to say input canvas load pixels and then i equal 0 i is less than w times w times four because there's four elements of the pixel array per oh no no no i'm gonna have to do the multiply by four somewhere else same thing because my input image array has w times w spots in it but instead of a random i want to say input canvas pixels i times four okay that should work because i could just take the r channel if it's if it's a full grayscale like black and white image i times 4 if the array is rgb alpha rgb alpha then the r channel is 0 4 8 12. and so if the i is going up 0 1 2 3 i times 4 is 0 4 8 12 et cetera so i believe this should work look at that look at that beautiful auto encoder it copied that shape perfectly now just you wait and see let r equals map mouse x which goes between 0 and width to between 0 and width this is the most brilliant code i've ever written in my entire life i'm gonna map this range between zero and width now hold on everybody just hold on a second i'm gonna blow your mind right now to between the minimum of zero and the maximum width but in case i wanted to change that later it's nice to have the map function in there and then this should be r which is really a diameter so let's call it diameter and we should see [Music] i wonder if it never got interesting so this is not working as expected hmm am i i'm definitely redrawing the image redoing the pixels into the auto encoder let's try a square just out of curiosity and also wait wait wait i have an idea the range was 25 to 200. so in theory like only range that it learned is uh between 25 divided by little w and 200 divided by little w um so that might help let me just say input if i'm going to do a square input canvas rectmode center let's see what happens here i'm like suspicious that it's not something is changing all right do i need to train the model i feel like the model picked up the average radius of the circle it does seem that way all right i feel like let me just go back to um what i had originally which was um i just want to make sure i'm using the right model so let's um i'm sure it is but let's go to um where's the model saved oh it's here today at 12th grade it's a little bit silly but let me um i'm going to put this back to what i started with today so i i'm just putting it back to binary cross entropy adam even though it kind of doesn't make any sense and a hundred epochs and i'm gonna once again train this model and let it go the full 100 epochs this time all right everybody the decoded images look like the average of all parameters half circle half square with a medium radius side i know um so i wasn't getting that issue before so i'm just going to let this kind of power through a longer training process and see what we get so this will be a new the way i have this configured is when i train the model it saves the model in the directory of the p5 sketch and the p5 sketch should should pull that up all right while this model is training and since i'm halfway through the live stream this is a perfect time for me to do my sponsor segment today's coding train is brought to you by curiosity stream thank you crosstalk stream now i'm just going to repeat what i said at the beginning of the live stream but those of you who i know most of you probably weren't here at the beginning so guess what i have something exciting to tell you about it is the best deal in streaming today so i am going to go up here uh whoops and show you something curiosity stream let's come back over here okay whoops wrong thing curiosity stream um i'm signing in here okay so curiosity stream is a amazing streaming service with thousands literally thousands of documentaries um all in so many um areas related to things that i do here on the coding train in particular my favorite part of kinds of documentaries that are on oh my god i've got to watch this one whale wisdom i love whales are the nature document documentaries um secret lives there's a whole bunch of secret lives of blank um nature documentation these are great for kids um science space and tech um infinite rainbows wait a second here i mean it's worth signing up for curiosity stream just for infinite rainbows alone how did i not know about there's only 22 minutes long in the time span of one single coding train coding challenge you can watch an entire documentary about what exactly are rainbows learn the science of i'm sorry this live stream is over it is done i don't know if the music is playing i'm out of here i'm going to watch this right now um but if curiosity stream isn't enough with this special one-time well it's not exclusive 42 off discount you also get access to uh nebula let me sign in here um nebula is a streaming service uh made created by creators youtube creators hopefully many of you might recognize many of your favorites here uh not not just bikes is new to curiosity stream i'm sorry new to nebula and i've been uh really enjoying not just bikes even though now i live somewhere where i have to drive everywhere so nerdsync is great um all of these wonderful channels um you can watch all of the videos from these youtube channels on nebula with no ads no sponsor segments and many of them have extended versions this is something i'm hoping to get into in 2022 so you could sign up now and so um the other thing about it is there are a ton of originals so if you look at the nebula originals you can see different um kinds of video content that you can't find look at this next officially you're in in gaming next level world building why games are better than movies is a wonderful nebula original series none of these these are only available through nebula and uh i'm going to take this from renee ritchie this is i just bought a dongle yesterday i needed a dongle for one of my laptops it cost more than an entire year it's like less than one dollar a month for both curiosity stream and nebula for the whole year it's 11.59 um if you just sign up through this link um it lets them it lets them know you found out about curiosity's stream and the nebula bundle through me helps the coding train out gives you access to all this wonderful content supporting educational content creators so i hope you will consider it um something that you think about doing for your streaming and entertainment and education needs uh for 2022 i don't know what your new year's resolution is but my new year's resolution was to watch more wonderful youtube con educational content creators on nebula all right let's check in uh back to our uh training uh by the way the link is in um a pinned message in the chat if you're looking for it's also in the video description okay so this is presumably the the new model was trained got down to a loss of zero point sorry for all the scrolling 0.644 i'm gonna refresh this page weird i'm like suspicious that my code isn't working because did i delete i deleted all of the let's go back to the random noise so first of all let's just check here oh wait a second no public yeah this is the model from 12 16 p.m that's the new model this project i'm not making any progress today at this project which is very distressing i feel like it did something a moment ago like when i switched back in um hold on we're going to debug this where's my next image okay i got to go back to the sketch next image function what if i go back to oh who knows what the problem is i know what the problem is come on come on chat come on you can think about it what's wrong what are the numbers coming out of inputcanvas.pixels if it is a white pixel what value is it 255 i didn't normalize the values i didn't normalize the values this needs to be normalized divided by 255 okay all right that didn't seem to change anything ah um okay let's just for the sake of argument put in random noise again so i am getting a variety again ignore the fact this is in the hmm i times 4 that's right right it's pixels should i give it it's not in the draw let's check the input all right but i mean let's just make sure it's continuing to run and make new input whoa no that's right 56 by 56 3136 okay but like a bunch of zeros yeah i mean this looks right all right let's try like why is the model not able to give me a range of sizes it was not having that problem in the version that i made where i was passing pearl and noise into it and look it's like what's it it's doing something okay use the mouse click to switch between square and circle am i getting alpha my i don't think i'm getting the alpha i times 4 should be the r channel all right this is a good idea so if mouse pressed mouse is pressed draw a square otherwise draw a circle okay that's definitely something is the scale wrong the mods that map oh no but it's symmetrical it could be that i'm not reading the pixels in the right order no it's totally symmetrical so i don't think that should matter all right here's the thing that i could try let's go back to our model let's let's look at okay so let's think about the model architecture for a second yeah many jimmy's saying accumulate the rgb and divide by three that shouldn't matter the mouse mapping seems a little wrong yeah i don't understand so hold on let's do this without the mouse mapping just to be more consistent here let me go here and let me say diameter let me make diameter global variable um start at the smallest and then where do i set it yeah and then i'm going to say just have it cycle back so i'm just going to have the diameter cycle automatically it's weird like this to me like the fact that it's not changing at all makes me suspicious that i'm not doing something correctly because in the neural network right like in the output folder in my test images i'm getting really um right this is this is what i'm getting when i'm testing it so like why is p5 not doing this so this is what this is why i'm suspicious because the model otherwise is performing as expected just not when i bring it into p5 so i feel like maybe the way that i'm drawing these images is somehow wrong all right i have another idea just bear with me for a second let me take a training data image oh this is no no let me take let me take 10 images from the training data and let me put those in a folder called data we're going to figure this out okay so now i am going to um joseph asked when you were using noise were you using the noise value or the pixel value oh i was definitely using the um the noise the noise the noise value hold on let me add preload so we're going to call these test images test images index i equals load image data what are these called square i dot dot png and the oh i need to use the back tick and this should be number format four uh and then i need a loop do you have 11 for some reason okay so now um let's just for a second take this out all this out i'm going to draw image test images zero now again i'm not let's let's okay so let's take a look at this oh you know what there's a little bit of an issue here i'm worried that i wasn't refreshing the code hold on don't worry okay so this looks better um let's get rid of this now um function mouse pressed let's have a test index equals zero so this is test index and then test index plus plus okay that's the first image okay see nothing is changing now is nothing changing because i am incorrectly i'm pretty sure that i have the cache disabled but let me just check under network is what i'm looking for yeah i have this setting should should stop me from caching the model file um but why so now i'm going to do this let's actually load the let's load the pixels of this image and this should be no this shouldn't be oh yeah i don't need this input canvas anymore this was silly right i can just draw that test image directly load pixels pixels i times four okay i'm reading the chat because okay ah okay so something about the canvas was wrong okay i knew something was weird well this is much more what i was expecting oh whoa boy did i just go off on a direction forever why what was wrong with the canvas input canvas pixels input canvas draw the image like what's the difference okay put this and just put this back okay so and now why why if i say input why if i do this instead of the test image the load pixel this must be a p5 bug this has got to be a p5 bug oh this is so sad but i guess this is good that i discovered this because maybe it would cause me some problems later why is the canvas can i not seem to get new pixels from it i mean one thing i could try to do is recreate the graphics context each time still not working is this asynchronous or something i commented out the update what am i missing right there's no difference here i mean i i guess to be 100 sure i need to draw the input canvas oh but but that's gonna cause problems if i'm recreating it so i have to put this back into setup yeah why why what i mean okay let's try this what if i were to say test image copy input canvas i mean i'm just like copying the image in extra time so right instead of using the pixels from the actual canvas i am i mean do i need to update the is it as simple as i just didn't update the pixels because i did load pixels that could be it well this works so i like copied the pixel from the canvas into an image but could it have been hold on this does kind of make sense all right this is what doesn't work i mean i have a workaround now so that doesn't work but what if i were to put input canvas update pixels no is that the wrong place for them i don't no what i'm doing no i think it's just broken i think the input canvas is broken so i'm going to go back to this ridiculous solution i don't this is working but now i'm going to go back to this and forget about the test images there we go and now in theory it's interesting how it's having trouble distinguishing the circle in the square but i'm not going to get too bent out of shape about that i kind of like that anyway yeah so i definitely eric is saying what if you train a data set that randomly rotates the square i 100 want to do that so i'm just unfortunately like i'm i'm this this is like the slowest thing i'm ever because i'm running into so many small little bugs but i have this working so what i would like to do is i would like to uh add the noise oh my god so just to finish close the loop on this uh [Music] and i see some comments here um in the discord chat as well so just to close the loop on this what i would like to do is let me add some noise so i'm gonna say i'm just gonna add 10 random dots so let's give me a random x and a random y and do a point at x y with the same stroke weight so how come i don't see those oh i don't need this last argument rgb uh point oh input canvas sorry input canvas xy so you can see that the noise is i mean this is like a sort of pretty terrible demonstration of noise um let's do the following um let's give it like a hundred points uh stroke weight um this is so silly that i have this like divided by two at w everywhere but let's just make it stroke weight one um and then also we can do the stroke like a little bit lighter [Music] oh i forgot to do input canvas that's why input canvas input canvas uh can i get away with like 500 yeah so this is what i wanted to show you of course we have this weird squirkle thing going on but this is how an auto encoder can denoise and let's try doing 1500 right the noise is completely eliminated and yet i am drawing a nice circle or square you can see it's a little bit more squarey as it gets larger a little bit more squarey wizzy says unsure what this is all about i just got here from the quadtree so first of all welcome i hope the quadtree video was helpful to you wizzy what i am doing currently is i am building and this is the diagram of it an auto encoder in uh javascript using both tensorflow.js and p5.js and what i'm now demonstrating is and and monos is saying this isn't really denoising the model wasn't trained on it so the way that i'm approaching denoising here is not giving it noisy images with a target output i'm just having it learn a singular kind of output and then if i give it a noisy image the only thing it knows how to do is make a copy of that input with a certain kind of style output thus removing the noise so that's really how i think of this as denoising um so this is exciting to see the interesting thing would be like what happens if i let the circle be bigger than the range in the training data set i assume it won't be able to reproduce that but that'll be kind of interesting to see so like for example right now um this diameter its range is going up to 200 but what if i actually like let it go from zero all the way to the full width so you can see as it gets to 0 it doesn't know what to make of that oh whoops oh sorry up this should be to this sorry so at a certain point it's going to get larger than any training image it ever looked at and it's going to get confused let's see what happens yep there that happened right there so it's like trying to make sense of that but it had no it can't extrapolate beyond what it learned but if i were to retrain the model with images that were the full size i'm giving it a bigger range of images now but if you input an image of mario brothers it will draw a squirkle anyway exactly it's going to draw everything into a squirkle and if i uh retrain the model now i kind of want to just for the sake of argument now now that i understand what wasn't working what the issue was let me go back to the model training code and i'm really determined to finish this project today so i'm obviously going to go past one o'clock but let me just return to this i am curious to [Music] see what happens if i try go back to mean squared error as the loss function and let's just change the optimizer to sgd i don't know if this should be better or not but let's retrain the model you can sing the model training song okay so i don't sgd doesn't seem to be maybe i need a higher learning rate with sgd um so let's go back to um i can't remember if i specify the learning rate anywhere i didn't i didn't i'm using a default learning rate but let's try mean squared error and let's see um let's see how well this does um the inside of the rectangle looks bent in this processing sketch so i think that's just an optical illusion i don't want to run this again although i can without saving the image i think that's just an optical illusion because of the rapid pace of circle and squares if i were to change the frame rate to like five i don't think you would see that traditionally we use noisy inputs while trading model rather than just a test set but this is perhaps the best way to do it given the simple nature of the problem thank you monas that's a very useful commentary all right so what's happening next while this is training um and let me just put this back um i want to go to the github repo for this and this this isn't going to work yet because it's still training the newer model although it seems to have like got stuck on a local minimum like which can happen from time to time i don't know if i should adjust the learning rate does anybody have a suggestion i kind of want to just run it again and see though it's it's stuck in that same spot where do i put the learning rate is that here so let's go to tf.js fit dot fit um call backs validation split sample weight initial epoch yield every where does the learning rate get ah it goes with the optimizer okay so if i wanted to try adjusting the learning rate would go here like this so let's give it like just curious like a super high learning rate oh whoa it found its way past that so it just took a little bit okay whoa okay i'm gonna i'm gonna let it this looks like a much better model with mean squared error now i'm gonna let it keep going and not worry about so i'm just gonna keep this in the back of my pocket if i wanted to i'm going to let this model i'm going to let this model finish training the loss is like very very very lower than anything i've seen to date yet working on this project and now what i want to do is go to the github repo for this and um i didn't i didn't see this one um sorry mini james but i want to look at this one so um this particular github issue was filed um [Music] had us has some really really helpful uh tips in it so first of all um i managed um so thank you so much to java gt who writes so this is how you can you can access the individual layers with autoencoder.layers so i made a little helpful function to split the auto encoder by looking for an increasing number of nodes so this is the way i'm going to be able to do that so but you can't just loop over these layers adding them to another sequence they don't carry the weights so i solve this with another helper function that creates a new dense layer so this is interesting to look at i suppose i can use this methodology um and um there's a lot of other helpful like tips in here about things so that's what i want to do next um also uh generating a gif like all this in the nodes in the node program make sure that decoder is fed with values zero through one um okay so but i'm going to come back to that i want to work on splitting the model next um but where is it um managed to get a 28 by 28 encoder reduced down to a single value couldn't do this at larger sizes which worked well down to four variables one makes sense because there's only one variable changing the the that radius um okay are we done all right we have a new model let's see how this new model looks so this new model should be much better yeah you can see like at the larger size it's really distinguishing the circle versus square and denoising now interestingly at lower at a lower size it's having a lot of trouble um it's having a lot of trouble um distinguishing the um the circle and the square because the bends probably of the circle are obviously much less extreme and harmony is doing a really great suggestion which is what happens when you translate the input image all right so what happens if i'm doing too many things at once here but i'm going to get to the latent space thing in a moment i have to make a new clearly make a new sketch here what happens if i draw that image at um at an x y position of my own design so let's see what happens here so you can see as i move it around it's confused but when it's centered that's kind of cool to see you can see like if i put it over here it thinks it's a it's like that must be a big square because there's dark pixels over there but maybe it's a small one and i don't know why i've lost the oh this one's supposed to be um circle no wonder so a good thing now what would happen if i trained it with in circles and squares that are anywhere in the image oh can i do that but i want to do the latent space thing if you use cosine annealing learning rate you can use a high learning rate the schedule or lower yeah yeah yeah so i don't know that i'm going to get that sophisticated with this but quick summary of the comment from manas is that you can start with a high learning rate and then over time like as it gets closer and closer to the optimal weights you can lower the learning rate so that it can fine tune it better but i cannot resist we must not stop let's try um having this also be let's try having the squares and circles in the training data set be in a random position and this has to be float because i'm not in javascript land anymore uh wait a sec something is wrong oh because i'm doing center no that should still be fine oh i'm resizing it down so this has to actually be these values should be the actual canvas size yeah all right this is going to be really tricky is it really going to be able to learn this convolutional layers would solve this i probably should not be trying to do this this is just asking for trouble but i cannot resist and remember my auto encoder only has eight units in the middle maybe i should give it 16. maybe for this i should give it 16. let's give it 16. uh i kind of want to save that previous model in case i need to go back to it so hold on um model squirkle now and then i think i might need to create the model directory again but it's empty okay so now i am attempting to see uh uh i think somebody might be here that i need to speak with hold on a sec everybody while this is training um sorry i'm just checking something here it might just be a delivery truck but i hear a loud noise oh maybe just retrieving something okay how's this going here oh no this seems kind of stuck i i think with many sizes placements you'll want a larger training set that makes a lot of sense the model size is less important than the data all right so that's a really good point okay i think i need to backtrack because my goal was to demonstrate latent latent variables so let me backtrack and not go down this road right now i think i could have done like if like a rotation maybe would have been so let's go back to um oh no that's wrong so let me put it back i had saved the model but i realized i need to deconstruct the model so um let me quit this let me just start the training again all right let's take a look at what i want to do so ultimately and let's put this back to eight i wonder if i could get it down to four oh let's do eight that's fine let's see if the loss does well with four okay so i'm trying to put it down to four just to see how we do oh i didn't make the new training data i have to make the new training data sorry everybody my brain has melted here uh what is going on okay let us go back to this wonderful let's go back to this wonderful code here so this is taking out the layers creating a new layer setting the weights okay so this makes sense i feel like there must be a way to do it um so i don't i think because it's such a simple amount of layers right now i don't know that i need i'm going to use this as a reference but i think i might be able to just even just save those layers in a variable and do this more simply um okay um let's see how this training is going wait what just happened oh yeah no the images are done okay i didn't take a break that's all kind of a problem um andrea says those sigmoid activations seem strange shouldn't you have only one after the final layer i think that's correct let's do that as well let's have all of them let's be able to expand it out only the last one should be sigmoid let's see how this does for us okay so now just examining just examining the code i know i just want the last three layers so let's look at this so now what i want to write some code is create a new model with just the the the decoder so i think that probably the way that i should be doing this though is rather than extract the layers shouldn't i just save them as i'm going like for example what if i were to say what if i were to create an array sorry for the noise everybody [Music] yeah i could do all right um all right so what if i i'm just gonna do this oh yeah i have an idea sorry couldn't i change do this and then have decoder layers and then basically do exactly this um decoder layers push yeah just the helicopter landing outside don't be alarmed he's picking me up i've gotta g i've gotta get somewhere fast uh all right so i'm gonna add all these decoder layers okay then i'm going to say 4 let i equal 0 i is less than decoder layers dot length i plus plus then i'm going to say add decoder layers index i um okay so now i put the decoder layers in a separate array then i'm adding them to the auto encoder so this should be the same i hate to just constantly retrain the model but let me just make sure this still sort of like looks right yep this seems reasonable so so everything is the same same as it was before now if i go back to this reference i want to create a new model so let's it's nice to put this in a separate function but i'm going to pull this code from java gt and see if i can kind of unpack it but with my methodology so oh i see but so i'm actually going to return both the decoder layers and the auto encoder as like an object because i might want to make use of both of these and then i can say do this so now i'm building the autoencoder but saving the decoder layers in a separate array okay um by the way banning if there's like spam happening in the chat is perfectly thank you nicole for stepping up and doing some moderation um i'm just gonna okay i don't know if it's possible to ban or not okay okay so okay so now i have access to the decoder layers separately so what i want to do here is create the new model with just the decoder create decoder with both the decoder layers and the auto encoder i don't know if this is going to work so now i just want uh create decoder is that what i called it decoder layers and the auto encoder and then i can go and take a look at this and say create the new model the decoder is a sequential model then for every for every decoder layer right this is the sort of i'm kind of doing the same thing so this there might be a lot of redundancy here so we can clean it up later refactor this later but i am creating the decoder creating a new model just by taking the list of decoder layers that i saved to kaya suzuki someday there'll be too many of these and i won't be able to do it it's good tsukaya suzuki welcome to the coding train you are boarding right now slow down train we're pulling into the station and opening the door and letting kaya on board for your coding train membership you win not this ukulele my prize ukulele but your very own random number uh on page 163 line uh row 8125 column one your sequence is get ready zero five three four eight okay um so close here so i'm adding all of the decoder layers to this new decoder model then i need to compile it i'm going to compile the decoder with the same settings that i used for the auto encoder so let me just make a 100 sure that i'm doing this correctly by actually copying this up here and just changing this to decoder again any returned in code that i can consolidate and do in a better way i will um but the issue is i believe maybe we don't have the weights so oh it's making a new layer why do i have to make a new layer and copy the weights oh is that really why can i just add the layer with the weights manually copying the weights huh so let's i feel like there's a different way to do this with um tensorflow.js but if this works then awesome so sorry i'm gonna do it i'm gonna i'm gonna follow this way so i'm actually not adding the decoder layer directly i'm creating a new layer with so the old layer is that decoder layer and i'm making a new layer with the same number of units the same activation function and the same input shape in theory i shouldn't have to do that because the input shape should be inferred but and then i'm going to add that new layer but i also need to copy over the weights like this okay and i don't need the auto encoder to be passed in because i thought i was going to need that to pull the weights please use four of loops you know what i would like to do that as well let layer of decoder layers yeah then i don't need this much better so this should be the function that and again i i feel like there's got to be a different way of doing this but i don't i'm fine with it if this works great um in which in where i am [Music] and by the way i shouldn't have to retrain the model now i should be able to just um pull in the auto encoder load it because i have it saved um but where did i save it is the question file public model so i have to do this so oh the last training got kind of stuck at like a not a great loss let's go should i just go back to 16 let's go back to 16 units in the center um oh this needs a lot of work 16 units or eight let's go back to eight eight is fine i can do eight we're gonna go back to eight um now i've added the decoder layers all right and then returning them both so now create deco okay so let decoder equals create the decoder and then i want to save decoder save decoder equals public file public model decoder so i'm going to save the decoder to a different directory okay everyone isn't there a model.getlayer method there certainly is marius so i i feel like there's there's probably a much more proper tfjs way of doing this but i think this should work let's see what happens unfortunately i'm gonna have to train the whole model again and hopefully we're gonna start seeing um i wonder if i do need to play with the learning rate or is it just going to break out of this somehow so let's take a look here i could just load the model i had earlier why why why why why why why why why am i just too impatient and i should let it what is the deal with it being completely stuck did i not put the new training data in there hold on let's make sure the new training data is in there yeah that's the new training data what is going on yep this is the new training data i'm not saving the decoder there's a typo in the code oh thank you well that's good that gives me another excuse to run this again i mean i'm tempted to just load the previous model i have that i know is working well and then but what let's just generate a new training data set again just to be sure lower the learning rate okay so to do the learning rate right now i'm just using the default atom so what would be a good learning rate i'm just going to try this point zero one uh now this is weird the loss is not going down hmm did this mess up something it really shouldn't have i just put the layers in here and then i'm putting them in the auto encoder one at a time this is not going well change in the activation function i don't think so i mean i can put these back hmm maybe it's thing with your layers array i know well now now i've now i've got it it seems to have caught so it all right i'm gonna go with this and let this model finish training because now the learning rate is and this by the way is now just with whatever the default learning rate is what is it tensorflow js default learning rate atom point zero zero i don't know if that's from 2018. so this is probably the default learning rate for adam okay we're still going down why is the accuracy zero let's just see how this model does it's so hard oh okay i like this epoch 85. oof what was that why was that one time that i uh got this incredibly low loss oop cannot read oh you know what i forgot to return it hilarious but that's fine let's just see now um let's go back to my p5 sketch why is this model so much worse all of a sudden okay so look at this model i mean it's definitely doing something what just happened i see the chat going a little bit crazy um but if i go here and load model dash squirkle look what what did i do for this model this model is working so well like this is what i train like when i train this model like what what happened i don't remember like what settings were different okay ah this is driving me crazy auto encoder 256 128 8 128 256. and these are the decoder layers that i then add one at a time just make sure for a layer of decoder layers add that layer then compile it mean squared error yeah i don't need this accuracy metric i guess just take that out and let's just be explicit about the learning rate so i can sort of see what it is i wonder if i got lucky with the train let's i could give it more images let's generate another set of training images use i'll give relu another chance okay but i want to use sigmoid for the last one because i want to uh i want to [Music] make sure that i'm bound between zero and one and maybe i shouldn't okay okay here we go everybody wait hold on let's just see here um this is like really awful how i have like this duplicate code but this shouldn't matter actually though because i'm not going to be training this model so i'm not sure why i need this but i'm going to okay okay okay everybody i want to give relu another chance model summary okay i like this idea i can't believe that i'm gonna be another one of these live streams and not gotten to the part that i wanted to go to want to go to save train model okay is this right what you're suggesting oh my uh yeah it's completely stuck learning rate okay like lower learning rate okay i would have thought i need to raise the learning rate but like to me i need to raise yeah i could [Music] lower the rate divide by 10 until it works this does not seem like what if i have a high learning rate eight eight three two is just like where it wants to be what is going on but what what did i do like did i change something in the training data i don't think so training data looks reasonable yeah interesting okay what just happened here no sigmoid no sigmoid a terrible sigmoid okay wait this is good hold on but i was like messing around with the learning rate okay i'll leave that default learning rate okay i think we might be in good shape now people i think maybe i think this sigmoid was not helping me i'm sorry i'm sorry in my head because i'm doing something that's the most sort of like basic simplistic like demonstration that i should go back to the classics like sigmoid but i guess that was really standing in the way of progress here standing in the way of the progress did i tell you about today's sponsor another hour curiositystream.com codingtrain are you asleep do you need to wake up as the coding train board you you could try watching something on curiosity's dream or nebula for less than one dollar a month for the entire year oh my god okay we've stopped no it's still going down little little bits and pieces little bits and pieces what did i do that one time that we got that crazy low loss was it just like luck drop the non-linearity for the last layer ah k weekman is here probably could have saved me like at least two hours of wasted time right the losses definitely seems to be nope still going down i want to see that like scientific notation though then i know i'm really doing something right okay oop hiccup a hiccup the number went up this is very exciting i mean needs really needs some kind of like something all right 256 128 8 128 256 3136 that's right total parameters trainable parameters non-trainable members okay we got the model summary might need more epochs or a higher learning rate well let's just see how this i refresh this should be oh no this is still whoa what okay this is insane i've not seen it do this wait but am i loading the correct model yeah look at look at this crazy model what it's like outputting so funny i was there was a time earlier today where my model trained perfectly and now what is going on uh k weekman this is our current setting um where oh sorry i mean the wrong code the current this is the current setting um yeah where's the noise coming from right yeah let's let's let's remove the noise at least in our input image just to see how well the model is performing otherwise um this is so weird the model was trained and has some like i think this is because of relu in the last no i mean where did that noise come from where are those dots come why are they there correct i know everyone in the chat is like 30 seconds behind me but the left is the input image the right is the output and i'm not sure where this new noise is coming from uh the dots are because of the rayleigh on the last layer negative values becomes zeros oh over 255 um okay so maybe maybe it's just an issue with me drawing it i have an idea hold on hold on i think there's an issue just with me drawing it so uh where am i where am i getting the values predict await output array output image render output image fill so um let me get that value and let me constrain it to between 0 and 255. no it's still there it's very consistent relu is in the last layer clip the values i thought that's what i was doing um this shouldn't make any difference all right what if hold on if val is less than okay hold on get rid of the constrain maybe what i don't want to do is constrain if val is less than zero val equals 255. let me just make sure i'm actually editing the right code okay i'm totally lost all right try sigmoid okay we're going back to sigmoid people so i want the last layer to be sigmoid and i want to just say atom here this is what i want i want this simplicity and this worked yeah um i mean do i just need more units i shouldn't why why why why why what's going on this was working two hours ago no problem it was training perfectly and it was reproducing it perfectly oh we can ignore this nonsense i mean maybe i should really be controlling the learning rate i'm reading the comments right now and i'm appreciating them i've got to figure this out shuffle i didn't change anything the only weird thing i did was like adding the layers through this weird other way but i don't understand how that could possibly be different and the model summary makes sense i'm so frustrated oh my god it's 1 40. this i have me i have a meeting i have to go to the input is a 56 by 56 image i had upped the resolution because why not i could certainly lower it back down just to like yeah i mean we could lower it back down until everything will run a lot faster let me try that i'll make a new set of images stroke weight at 8. well there we go there we go so okay so why was 56 by 56 freaking it out so much like it's really not that different of a problem to learn i guess it's just the orders of magnitude higher all right well i'm going to go with this just because so this has to change i got to change this to 28 which would be the only thing that matters this is done okay i must have some kind of scaling thing off because i changed it to 28 somewhere i'm not using these anymore where did i mess up this is not a thing this is not a thing i'm doing what is going on are my output images correct no oh no something is totally messed up when i changed it to 28 by 28 that's weird am i hard coding in oh wait no okay 28 by 28. where huh what have i done why did i destroy everything look at the chat regenerate images all right okay okay did i not regenerate the images i didn't regenerate the images is that really true i thought i did but i guess i didn't oh hi gloria gloria woke up oh but now i'm stuck again oh no i'm not okay [Music] okay okay everybody i forgot to generate them gloria come here come here girl you need to go outside huh come here do you want to say hi okay she does not love to be picked up oh come here oh you're a good doggy but she does let me pick her up okay oh yes look at that loss function that is a good loss function okay would you like to go back down okay go back down gloria okay oh god she is shedding like crazy okay okay all right well at least this is working now let me put the noise back in although i had it working at a higher resolution which was nicer oh i'm good i'm gonna lose my mind here let me put the uh points back in and put some noise back in and we're denoising again and we're denoising so now now i can do the latent variables the smooth version of relu is soft plus okay so simon i see your messages about the activation function um so i but i think i'm gonna have to investigate that for right now what i just want to do is see if i can get the decoder only to work okay everybody and i've got 15 minutes to do this so first of all did did i save the decoder yes so this should be the decoder and it has half the number of parameters so that makes sense that it's half the weights so now let's see if i i'm going to make a separate folder called decoder and then i'm going to take indexed index.html sketch.js put that into here and make a folder called model and that will go into here and then in the code where it is saving the decoder um [Music] so many things here i had so many ambitions of like some things i was going to do with this that i am definitely not getting to um if i am the the it should be public decoder slash model it's where it should be saved i just changed it to that now i should be able to go slash decoder this is going to break right okay this is going to break increase either the layer sizes or the amount of training data yes 100 right more layers and nodes but resolution is a matter of data size thank you so if i create some versions of this after today or anybody helps with this i'm gonna get back to a higher resolution but right now this i'm gonna i'm gonna close things because i don't know where i am this is now the decoder sketch and the decoder sketch does not need an input canvas an input it does need an output image which i'm going to make random and we're going to load model.json this is the decoder get rid of this diameter now next image is the i need to make it it's not an input and image anymore it should just be six the decoder should take uh let like the z vector the latent vector should have how many values in it did i end with 8 or sixteen i think i did eight right so this middle layer oh sixteen i i had sixteen okay i gotta redo this with less i wanted to get it down to four but fine 16. that's why i trained the model i have a trained model that works so let's just find out 16 values z index i equals random one so i'm just going to give it a random vector and then x test it's not really a test isn't really exactly right should be just the one z vector predict await and then everything else should be the same except i am only rendering there's no input image anymore i am just rendering the output image and it should be 280 by 280. so i have no idea if i did this right but let's see uh error due to auto encoder so decoder oh this is decoder square was expecting the number for the third parameter let's get rid of doing it again output print so it got a tensor okay weight output array it got an array of values oh i think maybe no i'm filling it at the beginning so let's let's not call this for a second ah w is 28 i'm going to look at the chat okay so what it's getting something it's getting 784 pixels i plus j times w oh wait oh let me no loop please sorry but this worked so where was the error and why isn't it drawing anything square is expecting a number for the third oh does w not oh my goodness so for all this sorry everybody i forgot about it's just that okay boy tiny little mistake there so now we should actually be able to all right so that's one image i got from the model uh oh am i supposed to put a weight here i think i might have killed this let's take out that print oh i still have some other crazy console log so i would have expected this model not to produce such like fuzziness but let's just out of curiosity if i give it like all point five yeah hmm oh but this is the input is like part of could it am i like oops no no no no interesting so the question is did i make a mistake somewhere yeah right okay so i guess i can't really i'm not really going to be able to sit onto saying this generate just as just using the d you have to re-parameterize the latent vector so i was going to just make sliders so my my plan was this follows um so take a look at this um uh sliders push create slider so the range should be between 0 and 1 starting value of 0.5 and an incrementation value of 0.01 and then what i wanted to do was have this do sliders index i value output image not being defined what did i do here oh line 34. what did i mess up oh did i lose that i still lost that by accident no oh let sliders equal an array oh not sliders index i sorry okay so i want to get this down to many fewer than just do like four and then see what i could i can see oh this makes it more square squared a circle this parameter yeah i know i know i need to use a variational auto encoder i just thought maybe the scale of relu is unbounded yeah so do i should i change that to sigmoid also would that help or like should i let them like the other thing i could do is just let the sliders have like a really higher range oops i keep doing this right i can get all sorts of interesting stuff all right so why don't i'm going to try the following and i realize i just i keep running this by accident so i'm going to let this run again and i'm going to go back to my model and i would like this i would like this let me get this down to eight i kind of want to make this four and then this activation should be sigmoid meaning these four units coming into here and i could add more layers but i don't know that i really need to um we'll all will be between zero and one yeah and i could look at some of the compressed images to see some values yeah but let's let's try this okay i'm stuck at a loss uh oh so close so close lower latent space i know would give worse results i just don't want to have so many sliders yeah it is good results so far so i is it going to help at all if i add some more layers increase the training data size oh wait oh oh it i just wasn't patient enough it like was stuck and then it caught okay hold on let me undo that let me undo that last little little extra thing i added okay now um go back to the sketch and it's still 16. it's still 16 but i used sigmoid yeah this is much better it's just too many latent there we go so i'm able to control all the different variables and i can kind of try to figure out which ones do which i'm trying to figure out how to make whoa there certainly is some extra weirdness here anyway this is exciting but i i really want it to be like eight so what if let me let me just it's two o'clock so i've got to be done i can't believe how much time i spent doing this um but this will be my last attempt i'm just going to like pump it up a little bit i don't know that this matters i'm going to give it an additional layer with 64. i'm going to keep the 16. i really think i should be able to get it down to 4. then i'm going to put it down to 4 and make that sigmoid then i need to add back in a 16 and a 64 right and should i make many more training images is that going to help me like if i make um 2100 i'm going to double the training images i really want i really want it to be all right let's do let's do 50 100 images i'm gonna get it down to four and i'm gonna let it train for 200 epochs um 200 image so this has to be 5 000 i know i need to make the images and 5 000. we run this i could also constrain the squares and circles to be less yeah so flubby everybody's telling me this really good suggestion which is to pass images through the encoder and look at latent vectors because that would certainly help give me a sense and that's a very important uh analysis that i could do right two should be enough one for the type of shape and one for the radius this is going to take a while i'm still just generating the training i'm just going to tell you about today's sponsor curiosity stream are you still watching this have you been watching this for three hours i can't believe how much time i'm spending on this auto encoder i hope that this will add to my life in some meaningful way this has got to be the last attempt and now that i have a model for this i can off on my own and you i'm going to push all this to github all of you can try tweaking this in different ways and let me know what configuration architecture of the model how many units you're able to get them to what i'm missing here i will accept pull requests because this is my last live stream one more line needs to be modified from one thousand to five thousand okay where's that i think i got both of them oh is it where i'm loading the images no i think i'm good i think i'm good yeah think of how many eternal rainbows you could all be has anybody signed up for curiosity stream and watched the rainbow documentary you could have watched it like 10 times crossing stream flat coating train okay okay let's try this okay okay patience patience everybody line 17 oh thank you [Music] got it [Music] oh this took a long time [Music] come on lower that loss you can do it lower that loss [Music] oh yes [Music] oh i like that i like that oh my god somehow i signed up for 200 epochs of this [Music] yeah so more data was clearly something i needed to do i also added more layers how long is it taking three seconds per epoch so i got like a little bit more so i got like 600 seconds left to go 10 minutes [Music] i mean i'm not going anywhere [Music] oh this is just the last live stream related to this auto encoder decoder project i mean i'll probably come back [Music] yeah i've got a really nice gpu sitting over here on the machine i'm using stream well the point of doing this was just like little bits of data [Music] oh that is a beautiful loss as soon as that scientific notation comes in i definitely do not need 200 epochs but i'm afraid to stop it [Music] oh yeah everybody while this is training you should sign up for the one year curiosity package it's less than it comes out to less than one dollar per month i'm not talking about less than one dollar per day [Music] at the local coffee place it's very expensive two cups of coffee for the entire year there you go two cups of coffee you know i get the oatmeal which costs 50 cents extra and then you can just watch this documentary about rainbows which is 22 minutes long over again you can tweet me and say thank you so much i watched the rainbows over and over again mark edward i haven't so my battery died simon [Music] ah pca yes code parade was doing projects like this uh so wait hold on come back see how this is going uh we're at epoch 63. oh why didn't i put in the lower north e box i just i could put in 150 epochs run it again but it's still going down let's get it let's let it look simon says oh sorry this i guess i could try to plug this in do i have a plug for it somewhere code parade was doing some projects like this the music was loud sorry about that i thought i turned it down um it always saves models every 10 ebox i know i don't think i put that in there i don't think i added any code to like save the model every so often and it's not improving anymore oh no there we go huge jump down um but simon is suggesting that i can use basically uh pca or principal component analysis to sort the sliders from most to least important um if you change your clock on your laptop it would already be finished don't think that's how it works hold on i can send some important text messages i'm going to be done in 10 minutes okay uh like the nice thing is didn't get very cold in here sunny out um where's gloria hopefully gloria didn't like pee somewhere because it's i've been taking her outside and this whole time i've been live streaming um are we i've lost track of what the loss is doing oh yeah it's going down further because i've got a if anybody could work out the sort of time travel i mean i just have to let it go at this point um i'll answer questions i don't know i could i could do some other work i could tell you more signing up for curiosity stream uh oh okay mark edwards today i just got here can i get a quick recap yes thank you for asking i'm gonna move over here just to have just to like get my legs moving a little bit so i'm building this is now my fourth live stream i'm building an auto encoder project and i'm using the tools of tensorflow.js which is a machine learning library in javascript and p5 which is a creative coding library in javascript that is good for like drawing and animation images i this diagram is completely sort of like not that useful anymore so me standing by it take that for what it is but um an auto encoder a great summary of what an auto encoder is you can find in this two minute papers a youtube channel what is an auto encoder and the idea is for a neural network to learn the um like a sort of like lower dimensional representation of an image that's a terrible way to explain it an autoencoder is a mechanism for taking an input image and copying it to an output image which is a very simple thing to do with just basic image processing algorithms like take every pixel copy every pixel but the hook here is that the auto encoder is not just taking all the pixels and copying them to the output it's sending all the pixels through a neural network that with each layer of that network has less and less and less numbers that it's allowed to work with so it's like an image compression algorithm and then a decompression algorithm can the neural network learn how to um sort of encode the represent encode an image into a smaller number of numbers and right now i'm actually trying to take these images of simple shapes they're squares and circles and encode them down to four numbers and really it should just be two theory we should be able to do two because there's only two variables it's either a square or a circle and it's the other variable is how big is it but i'm going with four so i have that working i've built all the code for it now i am training the model and i i just put in 200 epochs because i wanted to give it enough time to train and then when it's done what i hope to see this was like an earlier version of it is something that produces a much higher quality image than what you're seeing in terms of like it appearing to be a circle or a square and that i only have to play with four sliders to kind of manipulate it that's what i'm going for is there an epoch training dance no i'm so tired and hungry and exhausted and lost that i don't have one but if i could get it back it was only like [Music] this is how i feel this is my epoch training dance 164 [Music] 168. yes yes so mikhail is this i love this comment i still don't get how the compression works you've gone from shape and radius to shape and radius and a few hundred kilobytes of weights so to be clear i'm not actually doing anything of any utility whatsoever i'm trying to demonstrate the concept to and to help my understanding of how to architect machine learning models how to experiment with them how to perhaps make creative output with them and in a way like the concepts that i'm demonstrating with this very very basic scenario would hopefully extend how the music is would hopefully extend to use of say something like a more you know a gan or a style gan model sort of like looking at how generative models work thinking about how latent space and latent variables work i'm hoping that this whole process could um could provide something of note there but yes i mean i could write the code to just make the slider to a button and a slider and just manipulate them to get a circle or a square of a variable size um so there is um a coding trained group this is uh sa mckenzie asks i'm sorry if i tell me how to fix my name pronunciation um are you doing advent of code this year not so much personally although maybe now i might have some time to i'm just on my own but i don't think i'll be streaming it but there is a coding train advent of code team or whatever it's called and you can um if you're in the discord somebody please post the discord link you can find out information about joining and having your advent of code contribute to the leaderboard uh kobe who is the discord manager is organizing that so apologies for not knowing too much about it but you can definitely check into that yes so this is the older version with a less a poorly trained model and 16 latent variables when this finishes oh it finished oh my god so let's just look at this for a second 256 128 64 16 4. 16 64 122 okay that's right now hopefully oh it's a made-up name essay mcquenzy so it's a made-up name i don't know how it's pronounced great my friend seizure was great okay so now if i come back to my sketch that loads the decoder only i should be able to do this with just four sliders i think that's the only thing i need to change let me just make sure was this a new model from 2 16 p.m all right everybody i really should not do this any time and every time i've ever played this sound effect means it's not going to work but i'm about to hit refresh and hopefully we're going to see a nice squirkle there in the center four sliders that allow me to wait okay reading value undefined hmm at least it was just a syntax error ah i hard coded late in total equals four now that was anticlimactic okay i'm so excited oh let me tell you about today's sponsor oh no sorry sorry sorry okay refresh oh i like it i like it okay circle to square to square okay interesting so how what these variables do your guess is as good as mine maybe i um and i think i would love to do the random walk now but this is really really everything i ever wanted in an auto encoder latent vector exploration p5.js sketch i've only had an hour until i have a meeting in 10 minutes and i was planning to eat lunch in between this live stream and this meeting and answer all my emails and do everything else i needed to do so i can't do the random walk part uh because i would be like oh i have 10 more minutes i'll do it i have to just stop but um i will do that on my own let me push the code so the new things i'm adding are adjusted the data generation i guess this uh i added the the node cert the nodes code now trains the model and siphons out the decoder only i've got a new model and a new sketch a new model adjusted the sketch a little bit and then there's the new decoder and then i saved that earlier squirkle model so let me add all this live stream 4. um and if you're wondering oh if you're wondering um oh those models must be big if you're wondering where you can find all this um oh yeah it's here so uh i will accept i'm accepting now um pull requests on this um this was super helpful the way that java gt wrote an issue just to explain some improvements and some ideas i think this project is in a place where i can accept pull requests again i'm not at the moment looking to totally refactor it i i i need to write a message here because i've been i reply to this via in my like live stream many times i want to thank the chief here chief um for this incredible refactoring and contribution but it's too different than what i wrote during the live streams so i would love much rather link out to that but if you can help sort of optimize the sort of like architecture uh the learning rate um so that could get slightly higher resolution i think having four latent variables is good like little design improvements i would gladly take those um but i would very much especially love just documentation so images screenshots gif animations explanations in the readme it's not really your job to do that it's mine but if you're looking to contribute i gladly would take any form of contributions i'm going to put this to bed i think i've only got one more thing on the docket for uh 20 21 and then i'm hopefully relaunching the coding train and new in 2022 it will be the processing foundation end of year fundraiser i think that's going to happen now probably the 20 it's going to happen after christmas before new year's so uh stay tuned sign up for the discord thank you to um sponsor uh please check out i'm you know uh curiositystream.com codingtrain for full access to over a thousand over a thousand documentaries everything that's on nebula all the extra content so many wonderful educational creators for less than one dollar a month for the whole year it's 11.59 special 42 off discount check that out and um i really appreciate all of you sticking with me for this and um i will see you on the next live stream and i'm looking forward to all sorts of new content and community initiative now is the time to join the discord if you want to have a voice in the future of the coding train join the discord i would love to see you there um goodbye i gotta go i gotta i gotta have another meeting in like literally seven minutes i'm gonna do it from right here just i gotta sign out of this computer but hopefully i won't still be live streaming by accident it's the zoom call so i gotta i gotta play my outro music and all of that uh thanks everybody goodbye see you soon oh my god i cannot believe i cannot believe this day maybe i'll make the random walk happen uh just really quickly while i'm playing the outdoor music that's a good idea i didn't even find the outdoor music i don't even know what the outdoor music is anymore as always i always forget that this stop this stock this stop this stop i'm gonna do this stop this stop i'm gonna do this this stop this stop this stop i'm gonna do this stop [Music] [Music] this dot [Music] this dot song never forget this dot [Music] i'm gonna say once again here we go sing it with me [Music] it's look forward to cartesian coordination [Music] autotune and the internet will fix that for me take it with me to cartesian [Music] [Music] unicorns and rainbows and cupcakes what else is there yes kittens thank you very much kittens and rainbows and cupcakes notice that look what i get i'm really losing my mind okay let's do it [Music] and kittens the kittens getting some kittens and kittens the kittens the kittens the kittens kittens and kittens and [Music] i just quickly adapted this to move the sliders randomly which is sort of demonstrating a uh oh yes i wanted my stream called three minutes okay goodbye everybody that's hilarious i'm going i'm muting my microphone now i'm gonna let this play for another like few seconds just to the end of this song it's got one minute 30 seconds on the song bye everybody over again all sorts of text generation analysis things that i will use continuously over and over again first thing i need to do is yes okay we're gonna do it kittens the kittens the kittens and kittens kittens and kittens and kittens and kittens kittens the kittens and kittens and kittens kittens and kittens and kittens and kittens kittens and kittens and kittens and kittens kittens and kittens and kittens and kittens kittens the kittens the kittens have dusted [Music] you

Channel: The Coding Train

Views: 9,373

Keywords:

Id: A7eGllfV-iI

Length: 210min 10sec (12610 seconds)

Published: Mon Dec 13 2021