When A.I. Becomes Creative

this video was sponsored by brilliant.org hi welcome to this week's episode of cold fusion last week we took a look at electric flying cars and the problems that still need to be solved then we took a look at electric aircraft in general but today we'll be doing a bit of a fun episode we're going to take a look at some of the more creative applications of neural networks as well as some concerns the idea for this episode came to me while i was browsing twitter i came across some work by justin pinckney and doran adler their project turns any photo of a person into a disney pixar looking cartoon i immediately saw potential for such software and i think it could be a hit if they could polish it and implement it into some kind of app and i just thought it was cool so i reached out to justin and we had a discussion about many things and i'll include portions of the chat here let's get into it you are watching cold fusion tv [Music] so first a bit of background the current generation of neural networks is fairly new only being born in 2012 and in that time we've already seen them change the world before 2012 the computer science community thought that neural networks were a waste of time then a man by the name of jeffrey hinton showed them all wrong i have a full episode on that story if you're interested though one of the most recent impactful developments and what we'll be talking about today is the concept of a generative adversarial network or gan for short invented by ian goodfellow in 2014 gans are thought to be the closest thing that ai has to creativity and then today one of the you know things that's really taken the deep learning world by storm is your invention of gans so how did you come up with that gowns are a way of doing generative modeling where you have a lot of training data and you'd like to learn to produce more examples that resemble the training data but but they're imaginary they've never been seen exactly in that form before when i was arguing about generative models of my friends in a bar something clicked into place and i started telling them you need to do this this and this and i swear it'll work and my my friends didn't believe me that it would work but i believed strongly enough that it would work that i went home and coded it up the same night and it worked so take you one evening to implement the first version of cancer it i implemented it around midnight after going home from the bar where my friend had his going away party so how does a gan work basically it's a pair of neural networks that compete against each other by trying to fool each other and they both get better in the process there are two parts to again a discriminator and a generator let's take image generation as an example the discriminator learns to tell real images apart from fake images created by the generator at the same time the generator uses the information from the discriminator to learn how to produce images that the discriminator is not able to distinguish from actual images both neural networks learn together and get stronger throughout the training the training usually finishes when the discriminator classifies a fake image as a real one justin explains further the thing that's discerning between the real and the fake images keeps on improving and improving and in doing so it helps the generator to improve because it's pointing out what what the flaws the images are that it's general that are being generated and that helps the generator to improve those as you can probably imagine this is a pretty powerful concept in the ai space anything to do with creating and not just simply recognizing and predicting is probably using again and recently many tools have become readily available so there's been massive development and accessibility so let's take a look at some applications to get you up to speed here's a few things that gans can do we've covered some of this previously but many of you are new here [Music] gans can do things like animating the mona lisa and a more recent example includes turning line drawings into people this was recently covered by the youtube channel two minute papers how about a game that estimates what roman emperors probably looked like 2 000 years ago just from their statues a very interesting tool for historians i'd imagine interpolating choppy footage into high speed slow motion is also a common use they can also enhance grainy or low resolution footage to 4k with high frame rates check out these enhanced videos from the turn of the century absolutely amazing and i think this is a great use for such technology gans can also reconstruct 3d models of objects from images or how about reconstructing an image of a person's face just by listening to their voice in medicine gans have also been used to generate new molecules for a variety of protein targets associated with cancer inflammation and fibrosis in 2019 these gan generated molecules were validated experimentally in mice i'm not sure if this next one is again but it is very interesting nonetheless a neural network called jukebox by open ai can generate songs that don't exist you give it a portion of a song and it's going to try and make up the rest with some interesting results [Music] oh [Music] a key moment for gans in the general public came from nvidia in december of 2018. it was called stylegan it was a leap forward in image generation and moreover it was accessible in february 2019 uber engineer philip wang used the software to create this person does not exist.com he himself was amazed and stated that it was impressive that stylegan could quote pick apart all the relevant details of human faces and recompose them in a way that's coherent end quote i think that's objectively true you can try it for yourself at this person does not exist.com every time you refresh it you get a new picture of a non-existing person [Music] in february of 2019 stargand 2 was released and it came with higher resolution and less artifacts out of style gun 2 came our main story for today a concept with the working title tunification the idea is that you can upload or take a photo of yourself or someone else and the ai will create a disney pixar version let's take a look at some behind the scenes work that justin has been involved with while developing tunify what we made in the end was this kind of tunification which was to take a you know a real picture of her face and kind of give it the characteristics of something that you might see in a disneyland pixar animated film so you know like bigger eyes and smaller chin and like flamboyant hair and that sort of stuff um and that's kind of the yes is that sort of moving beyond the basics of gans because you know normally gans just generate images from an existing data set but there's no existing data set of like photorealistic versions of disney characters you know this is something that's a little bit in between a disney character and a photo of someone's face and so that somewhere in between is you know normally quite a hard thing to achieve with a game because it's not really designed to do that but it's built out of these different layers which address different resolutions in the image so the lower resolution layers are all in charge of you know controlling the pose of the head or the overall shape there's like the middle resolution layers which are like the features like the eyes the shapes of the eyes and the mouth and the nose and then the highest resolution layers in there are all in charge of controlling the lighting or the coloring of the image um so because there's this kind of control over the different layers of resolution then you can actually do a thing where you combine two different models that you've trained so they have to be linked like and that's what we did for this unification project we took um one model which is the kind of standard thing which is the photoistic face generation model that got trained by nvidia and then another model produced from that um by someone who i met on twitter called doran adler and then swap some of those layers over so we take the character model which gives you the kind of big eyes and the shape of the head which is kind of characteristic but then preserve the high resolution layers from the photoresist model so that you know things come out looking looking a bit more like a photo and those kind of two things combined to give you this this kind of weird slightly creepy um real but not real image which is sort of in between the two in between those two things if you're interested with getting started with neural networks you may be interested in this episode sponsor brilliant brilliant is a great online platform and app that allows you to actively learn the maths and science that governs the world around us they have a few courses on neural networks everything from the history of the technology to how they work you can take a quiz at every point in your learning journey i took a course on the mathematical concept of infinity just for fun and i found the brilliant quizzes quite entertaining so if you're looking to sharpen up your critical thinking or planning on a career change why not do it in a fun way with brilliant to start learning today go to www.brilliant.org cold fusion and the first 200 people will get 20 off their annual premium subscription asked justin about what he thought the future of ai or neural networks will look like he's stressed on the point of efficiency doing more with less training these things with much less data than you need so you know just having thousands of images rather than hundreds of thousands or millions of images um and also making these things more controllable or smaller or more efficient and all of those things you know are useful for people who actually want to try and put these to practical uses because although the research side you know the cutting edge side of things is is really interesting you know a lot of those things aren't always necessarily very practical to use on a website or a mobile app in may of 2020 nvidia researchers taught an ai called gamegan to recreate the game pac-man simply by watching it being played this ai was trained on 50 000 examples of pac-man in order to produce a fully functional version of the game without a game engine this is impressive but then once you start comparing it to the human brain it's pretty terrible for a human we'd probably need to watch one or two rounds of playing the game to understand what's happening and subsequently recreate the game fifty thousand examples is way too much if we could train ais with very little info this could be the next big step another example is nvidia's deep learning super sampling this can upscale video games reducing gpu load so games can run at higher frame rates and higher resolution at the same time super computers to take a lower res image and then intelligently build it up to look like a native higher res so if you're playing at 1080p with dlcs set to performance or quality then the game is actually running at 540p or 720p respectively and then intelligently up scaling back up to 1080. for example at native 1080p this chain-link fence looks a little ragged but it's obvious what it is move to dlss 2 though and everything is much clearer compared to the native we've actually gained detail particularly on the fence and the handrails and we're still up at 77 fps from the native 43. though for this they had to use a supercomputer to train the ai later on in our chat we went onto the topic of deep fakes and the problems that they present good evening my fellow americans faiths has ordained that the men who went to the moon to explore in peace will stay on the moon to rest in peace for every human being who looks up at the moon in the nights to come will know that there is some corner of another world that is forever mankind yeah so i think it's an interesting thing and i think the you know there's a lot of research going into detecting the fakes and you know obviously there's a lot of research into kind of producing ever more realistic images and my feeling is those those two things will be kind of competing forever in a you know kind of a never-ending battle that one will get better for a while and then the detection methods will get better and then the generation methods will get better and that's probably just going to be the state of play kind of forever um and i don't really think necessarily one is going to win out and in that sense i feel like it's just something that people have to learn to deal with nvidia has a built-in method of detecting if an image is real or fake and other companies are building such tools for video as well in the end i think we both agreed that it will end up to be something similar to an arms race [Music] so there's been a huge amount of hype and people are really you know amazed by some of the results i'm really blown away by things but i think it's quite easy to to get carried away with that and not realize that a lot of them are still quite limited or you know when you get outside of their sort of domain of expertise they can fall over really easily that you know they put too much faith in them and they start to make bad decisions or bad predictions about things and if you don't really know that you just have blind faith i think there's a you know there's a big trap of just assuming that it's doing a good job and not really using any of your kind of human intuition or knowledge about is that really working very well so that's about it from me so what do you guys think about neural networks and gans in particular what was your favorite application of gans do you think that gann technology can bring a lot of positives to the world or are you more weary of this kind of technology let me know in the comments section below thanks to all of you who supported the channel by buying the first edition merch you should be receiving your voice messages and music shortly a standard line of merch is still available if you'd like to see anything on science technology business or history definitely subscribe to cold fusion there's a lot of interesting stuff on here next week's video will be on some research of nanoparticles that dissolve plaque and arteries and may stop heart disease my name is tagogo and you've been watching cold fusion [Music] cheers [Music] you

Channel: ColdFusion

Views: 813,450

Rating: 4.9448757 out of 5

Keywords: Coldfusion, TV, Dagogo, Altraide, Technology, Apple, Google, Samsung, Facebook, Tesla

Id: KZ7BnJb30Cc

Length: 15min 40sec (940 seconds)

Published: Wed Sep 23 2020

Reddit Comments

[u/victor_knight]

It's entirely about GANs and doesn't even touch on the actual sub-field of AI known as computational creativity.