'How neural networks learn' - Part I: Feature Visualization

one of the biggest remaining challenges in AI is that most machine learning models that we train are uninterpreted that means that when we train a neural network it is really hard to figure out why it's making certain predictions this is okay in most cases but many applications like self-driving cars or healthcare or cybersecurity they're very crucial and they require us to actually understand why an algorithm makes a certain decision and so to address this really important issue of interpretability I want to bring you guys a series where we will discuss how neural networks are learning and what they end up learning after training so the first part of this series will focus on feature visualization and various ways that we can look into a network after it's been trained and actually look at what it ended up learning while training on some tasks in the second part we will look at various examples of how neural nets can be easily fooled to make the wrong predictions so here we will talk about adversarial examples that clearly show that even though these networks are loosely inspired by how our brain works there are clearly some very different things going on and then in the final part I will cover a few very recent papers that discuss the trade-off between memorization and overfitting versus generalization strengths on unseen and new data and how all of this relates to you know the learning process that's going on in neural networks are you ready to dive in deep my name is Xander and welcome to archive insights so imagine we have a trained convolutional network that can do image classification the problem is if we want to understand what is actually going on in our network the only thing we see is a whole bunch of matrix multiplications and a lot of math going on so very very difficult to interpret that result meaningfully now luckily there are methods available to look inside that magic black box let's start with feature visualization so the simplest thing you can do for future visualization is simply look at your network after training feed it a whole bunch of images and then look at the activations inside your network if we take one specific neuron out of our network for example and we feed a bunch of images through our network most of the times this neuron will not fire but in some cases when a particular pattern is present in the input image our neuron will fire and we call this an activation and basically this neuron is saying you know hey guys I saw a very particular pattern in the input and that might be useful for classifying this image in a given category so take a look at these images for example what we've done here is we've taken four neurons inside a deep neural network and then we've looked at all of the training data images and we selected nine images for every single one of those four neurons that maximally excite those neurons and from these samples we can already get a really good idea of what a neuron is trying to detect and so before we go any further I want to know that this approach is not only limited to visual features so in this fantastic blog post for example by somebody LeMond who now works at google deepmind a very similar thing was done for audio let's take a look so the whole goal of this project was to see if you can use deep neural networks to do music recommendation so traditionally music recommendation is done with what we call collaborative filtering and this means that you're simply going to look at what a user likes so you're going to look at the music that someone listens to the playlists and their likes and then you're going to look for similar users that have a very good matching of those likes and playlists and what you're going to do then in collaborative filtering is simply recommend songs that person a likes to a person B that is very similar in music style but there is one problem with filtering the problem is that if you have an entirely new song there is no way to know who likes that song and therefore you cannot even apply collaborative filtering we call this the cold start problem and one solution to this problem is to use a deep neural network to take a song and map it onto the same embedding space that you would end up while using collaborative filtering and so the whole idea is that you take your song in audio form you map it onto a spectrogram which basically transforms your audio file into an image and then you use a deep convolutional neural network to map that image to an embedding space that you would get while using collaborative filtering and if you can do that then you can take any new song that nobody has ever listened to you can run it through your deep neural network and you get a representation of that song that you can then use for recommendation and it turns out that if you train a deep neural network to transform an audio song into the same embedding that you would get using collaborate filtering that this deep neural network learns some very interesting features that are present in audio so here we're going to do exactly the same as for the visual features we take one specific neuron and then we're gonna run a whole bunch of songs through our network and we're gonna look at which songs maximally excite our selected neuron so the first feature on the top left for example learns to recognize women singing vibrato tones if you listen to a couple of the samples here this is very very interesting because all of them have a very similar kind of feel to it and you can sort of understand by listening what this neuron is trying to look for caress me another neuron here for example has learned to detect the drums in drone base songs kind of cool right so while these examples give us an intuitive idea of what a network is trying to learn it can be a little bit confusing because if we look again at these examples here if you look at the dog faces for example well it's kind of hard to tell is this neuron trying to detect dog faces or is it simply triggered by a pair of eyes and take the one image on the right for example is that one looking for a sky in the background or is it detecting buildings in front there really is no way to tell for sure if we want to figure out what our network is really learning will have to do better so one of the first influential works on feature visualization was a paper by Matthew Siler and Rob Fergus in 2014 so what they did is they trained a normal CNN to classify images but at the same time they also trained a backward-looking network that try to visualize what the features were learning so here's how it works you run an image through your convolutional network but for every activation of a single layer you're simply going to take that activation and use it to reconstruct the activation of the previous layer we call this ad convolution operation you can train this with normal gradient descent using a very simple reconstruction loss so to visualize a given feature after training all you have to do is feed an image to the network pick one activated neuron then set all the other activations in that layer to zero and use the trained backward Network to map that feature all the way back to the input pixel space so instead of visualizing examples of images where a certain feature must be present using this technique we can really focus and accentuate the specific parts of an image that activate a chosen neural and this gives us a much more detailed view of what's going on so let's have a look at some visual results so here we can see a bunch of layers from a deep convolutional network being visualized where the first stack of images shows the reconstruction in the input space and the second stack shows the actual images that were fed to the network so in here we can already see a lot of what's going on we can see that the first layers in the network they detect very simple things like lines and edges and then the consecutive layers what they do is they build on top of those very simple features and they build more complicated structures and all the way at the end of your network you have very complicated features detectors that are combining all the information from the previous layers and these things are detecting features that are very useful to classify our images into the final output labels I want to note however that using this technique you have to feed an input image to the network so there is no way to visualize a feature without feeding it a specific input image now since the very early work on feature visualization there have been a couple of very successful new techniques one specific approach uses gradient descent itself on the input pixels to generate images from scratch that maximally excite a chosen neuron let's see how this works so we start out with a randomly generated image every single pixel here is a random number and then we're going to feed our image through the network we're gonna pick one specific neuron that we want to activate and we're gonna compute the gradient using back propagation and basically this says well how should I change the pixels of this image to activate that neuron a little bit more and so by adjusting these input pixels with respect to the gradient of that specific neuron if we loop this for a couple of times we can actually generate an image from scratch that is very very good at activating that specific neural so here we can see an example of what happens when we do that for a couple of thousand iterations and so the main difference here is that we don't need an actual input image we can simply pick a neuron and then generate an image that would maximally excite it and by being a little bit creative we can also create images that minimize the activation of a given neuron so here you can see a couple of results where on the left side you know there are images being generated that minimally excite this neuron on the right side we try to maximize its activation and then in the middle you can see examples from the training set that do both of these things so the results here are really interesting to look at and going a little bit further we can even generate images that excite two neurons at once really cool right and now before we go on I want to mention that even though these generated images they look really nice and you can sort of feel what the neuron is looking for actually generating these images is not as easy as it sounds it's not just backpropagation there is a whole bunch of different regularization techniques that have been tried and it's mainly these regularization techniques that affect the way that your final generated image will look like so for people that are interested in knowing more about the technical details I would definitely recommend to check out this blog post by this still because it is amazing alright so all of this work on feature visualization has led us to this very common and intuitive understanding of what deep neural nets are doing the first layers they recognized very simple patterns in the input data and then as you progress through the layers of your network all of those very simple features are recombined in different ways by the neurons in the next layer to get to more and more complicated feature maps and at the end of your network all of those feature maps are complicated enough that they can be used to detect specific classes in the images and so over the years a lot of people have been working on feature visualization and I want to highlight one specific work that I found really cool it's the deep visualization toolbox and so you can go through the github page and basically what they build is a system that you can use to analyze what a deep neural network is seeing in real time so you can hold a couple of things in front of the camera and you can actually look at all the different layers in that network and see their future activations in real time as the network is analyzing the input images a couple of really interesting things to notice is that here for example we can see that the network learned to detect text in images but an interesting note is that it was trained on the image net data set which doesn't have an output label for texts so what this means is that our deep neural net has learned to recognize text in images even though that text itself is not an actual output class and so it turns out that if you look at the training data text is actually very useful to predict classes like bookcases or traffic signs and being able to find text in an input image can be very useful to label such images and to finish off I want to show you guys one cool example of what you can do with feature visualization let's look at this very cool project by a Google engineer called deep dream and so what happens in deep dream is that you take an input image and then you're going to feed it through the network pick a couple of specific neurons and then you're gonna use gradient descent on your input pixels to gradually adjust the image in order to activate those chosen neurons as maximally as possible we see here if we start by picking neurons in the lower layers that we actually get enhancement of those features that the first layers are detecting so very simple things like edges and lines but if we pick features in the higher layers we can start recognizing specific objects like faces and eyes suddenly appearing inside the image and that's why they call it deep tree and so you can see that by picking features visualizing them and messing around with the code you can create some pretty cool examples all right so let's recap what we learned if we take deep neural networks and we train them using back propagation on some data set they end up learning very very simple features in the first layers and then consecutive layers will start recombining those features until we get very complicated feature representations that can be used for the task it is trying to learn the easiest way to visualize features is to just look at training examples that maximally excite the chosen neural a better way is to generate images from scratch using a gradient signal to maximally excite a chosen neural and the advantage here is that you don't have to deal with all the correlations in your input data you can just visualize a feature itself but when doing this regularization is key alright so that was it for to visualization in the next episode we're gonna have a look at adversarial examples and these are images that we can generate but when we give them to a neural network they get completely misclassified into the wrong label and while this is not exactly what we're hoping for these things can give us some very interesting clues on how we can still improve these neural networks I thank you all very much for watching and I hope to see you again in the next episode of archive insights

Channel: Arxiv Insights

Views: 79,319

Keywords: Feature Visualisation, Neural Networks, Deep Learning, Backpropagation, DeepDream, Music Recommendation, Understanding Neural Networks, How neural networks work

Id: McgxRxi2Jqo

Length: 15min 0sec (900 seconds)

Published: Fri Dec 15 2017