This is kind-of a follow up to Brais' videos on deep learning So deep learning is kind of a big thing at the moment and there's some disagreement between research over whether this is gonna be - the, this is *it* This is the big thing that's gonna change everything or whether this is another flash in the pan, like artificial neural networks were in the 80s Everyone got very excited and they got quite a good results and when they realized that they couldn't solve all the problems with them, I don't know For what it's worth, these are a big deal, I think [offscreen] Let's talk about convoluted neural networks, have I said that right? Convolutional neural networks. [offscreen] Ah, right, ok They combine both deep neural networks, which is what Brais was talking about and kernel convolutions, which is what I talked about in a previous video. I would thoroughly recommend people watch that video You know, it's got an entertaining host, right? *laugh from offscreen* So, but, but, because, if you don't know what a kernel convolution is, this isn't gonna make much sense to you So watch that video first [offscreen] So that's the kernel convolutions we did on graphics and things like, uh, sobel op- Yeah, Sobel operations, Gaussian blurs, and things like this. Sobel operations in particular, and edge detection So, if we think back to a traditional artificial neural network ok, what we've got is we've got some kind of input we're trying to learn, ok we've got some hidden layers, alright, and then we've got some output layers maybe just this one, I don't know And these are fully connected, so we have lots of connections from here and here and here and these are connected to here and so on, I'm just drawing in a few of them, and then these are all connected to the end. ok, Now using Brais's analogy, we were talking about house prices. ok, so, this will be something like number of bedrooms, and this would be something like "has it got a pool" and this would be, you know, what floor space is it and has it got a good garden and so on lots of these, ok, lots of inner nodes that we don't really care about particularly, or I don't Uh, right, and so on and so forth, and then finally at the end we have a house price. Now what this house price is, is a complicated function of these inputs. It's complicated because this node here is some linear, or non-linear now, combination of these. ok, so, a bit of this plus a bit of this, plus a bit of this, plus a bit of this through some non-linearity function. This is a different combination of these. This, again, different combination of these, and so on, right And then this, a different combination of these So, you can see, you're building up some kind of level of abstraction here where you've got combinations of combinations And that function is very complicated When Brais talked about a black box, in some ways that's exactly what it is because we can't look at these individual weights and say, "well that's got .2 of this one, so that must mean this" because we just don't know what it means, right In the grand scheme of things, in this whole network, we don't know what that individual weight means And to be honest, we might not even care that much What we really care about is how well does it predict house price, how accurate is that based on that for a different input, so we change this, read the output - is it good? Yes? Brilliant! ok, now, for images, which is obviously what I spend most of my time around, this is a start, but it's not very useful to me. If you think that this is our inputs, ok, and I give you a picture of a house, and I say "right, tell me how much this house is worth" ok, well, what? So how do I, ok, so there's two things I could do, right first of all is I could try and calculate things like number of bedrooms and stuff, based on the image and put them in here In some way, I'd be calculating some features and then I'd be putting them in here and learning on those features. That is quite a smart way of doing it, because, apart from that's obviously quite difficult um, it's smart because we don't have to have that many more neurons In anything, we can actually use the same network as we used before for our model on our house, all we have to do is work out the bit of code that does the image analysis Now, anyone that's tried to find out the number of rooms in a house based only on one picture of the outside of the house will tell me that that can't be done, right That's hard. Ok, so you could naively think, what we could do instead is just put the image in here. Make this the first pixel, and this the second pixel, and this the third pixel, and so on, ok. Then, this has got all the information it could ever need, right But it, but that's the problem 7 megapixel image, that's 7 million input nodes, let's say we have 7 million nodes on the next layer each one connects to each other one, you can see that that's just gonna melt my computer it's not even gonna try and create it, it's too much information That's why we downsample our space a little bit. What we would usually do is calculate some small subset of features and then we would put them in at this end. So that's quite important. So, traditional machine learning is done a bit like that. So Michel's done some videos on this. Calculate some features about someone's face and put that in to some machine learning algorithm What you don't do is try and put the machine learning algorithm just on the face because it's too much information there Until now, ok, right? That's where convolutional neural networks step in. So, convolutional neural networks replace each of these nodes with a kernel convolution. So, like, a Sobel edge detector Now, so instead of what I would've done before, which was run a Sobel over something and then machine learn on that, I just give this the opportunity to learn which features are interesting maybe it is an edge detection, maybe it is a corner detection maybe it's something that highlights whatever's in the middle of the picture Or something that highlights the top left-hand corner it doesn't really matter, and the point is I don't know what they are right, if I give you, you know, two thousand pictures of houses and ask you to predict house prices based on the pictures I don't know for sure, I can guess, but it might be - that how many windows it has and things like this but I don't know for sure. And a computer can brute-force through those things much quicker than I can and tell me And then I can go, I can both predict it, and I can look back and say, "oh, it was windows after all" So, let's imagine that what we have is our image, ok, so I'm gonna move away from the house analogy now because I'm gonna have to draw a lot of pictures of houses if I do that. Ok, so let's talk about CNN works Um, and why it's useful. So, we have an image of something Now, I have seen convolutional neural networks used for non-images but for now, we'll just talk about images This is a picture of, let's say me. It's, you know, it's not a great likeness but I'll stick by it Now, there are three channels here, ok. So this is actually a 3D volume, in some sense remember when we talked about 3D images, you could view RGB as a, in some sense, 3D So, the first plane is our R, G, and B, or vice versa What we do is, if we performed a Sobel edge detection on this, what it would do is produce another image that was slightly smaller than this and only one deep. So hypothetically, it would be another image where the edges, let's say the horizontal edges, were highlighted So it would kinda look like, that, or something, I don't know some half of my face where the horizontal edges are highlighted, ok It's not a great diagram But there would only be one output, because Sobel just outputs a number between 0 and 255, as soon as you scale it, ok Now the problem is that I don't know that Sobel's the best thing for this task, ok It might be, right, it might be useful to detect edges on houses, to work out what their prices are or if you want to detect faces, to detect the size of the face, that kind of makes sense On the other hand, it's gonna produce a lot of erroneous bits if I was sitting in front of a tree, there's gonna be loads of edge stuff going on there that I don't care about In a convolutional neural network, what we do is we do, let's say, 60 of these on the first layer. So we have one, and then behind it we have another one, and behind it we have another one, and behind it we another one, and so on, going this way. So the first one will be some convolution process applied to this whole image that takes three input channels and outputs one output channel The next one will be a different kernel convolution operation so each of these will have a different kernel those are our weights, those are these values here In sort of our analogy back to normal learning Um, and so let's say we have 60 of those, or 64 of those One of them might be detecting edges, one of them might be detecting corners Um, and then we use them as our features for learning Now that's a start, but we're - this is is deep learning now, right, so what do we do now well, what we now do is we do more features based on these features So we find combinations of corners, combinations of edges, that make something interesting My face is not just a circle of edges, what it is is a number of corners and edges and bits of texture and things all in a specific shape that is unique to, uh, well, certainly to a human face, but even unique to me right, because we're capable of distinguishing between different people So, this kernel window will go down to this pixel here ok, so this will slide about this image and produce this output image and then the next one will do the same, and the next one will do the same Then we do the same thing on this one let me do it in a different pen so we can see better. Here's my red kernel convolution and this slides about and produces another image, which is a slight combination of, maybe, corners and edges or something. I mean, this second level, it's not gonna be too abstract, but we'll get the idea So there's gonna be some sort of shape that's gonna be sort of... It's not gonna make much sense to us, but it'll make some sense to this machine. And there'll be another set of these, so there'll be lots of these, right, going back All of these will look different and be some different representation of my face transformed in some way, to be useful And again, I haven't picked these, these have been learned, just like a normal deep learning algorithm So I haven't had to say, "I definitely think edges are important for this" cause I don't know for sure. So this goes on, and we keep doing this, and sometimes we also downsample the size of these images, just to save memory, ok, but we won't dwell on that too much. And, because of the way that we downsample, and the way that sometimes these convolution operations slightly shrink the image, cause they don't go all the way to the edge, right If you've got a 5 by 5 kernel, you can't go to the edge 2 pixels cause you're going off the edge so we don't worry about that, we just get slightly smaller In the end, we end up with a much smaller image, and lots of features going all the way back So these are my different convolutions of convolutions, of convolutions, of convolutions And each one will look different, and represent something different and we don't know what that is. So this one, could be highlighted when it's a face in the middle, or it could be dark when there isn't This one might be highlighted when there's an ear at a certain position, and so on. Eventually, these will get down to being just one pixel, and very very long So essentially what we've done there is we completely removed the spatial dimension. There's no more spatial information left, we don't know where anything is. But we know what it is, because it's listed in all these features. These now are our neurons at the end. So we have a couple more layers that point to these, and then finally, we have one at the end that says "Is this a picture of Mike's face?" And it produces a 1 if it is, and a 0 if it isn't. And then what we do is, just like a normal network, we train it. So we say, "here's a picture of me", ok , so this should be a 1. And let's say it's 0.5. cause it's kind of random. So we adjust these weights, and we adjust the weights inside all these kernel convolutions. [offscreen] So does that adjustment happen manually? No, it happens using a, uh, well, it's coded in, um, but it's usually performed by a library, and it's using a process called back propagation. So what we do is we basically predict what direction we have to move the weights in to improve our output, and then we move them over slightly in that direction. And we have to do it in reverse order, because these ones depend on these ones, depend on these ones, and vice versa, what we do is we say, well, look, given that I've said it's 0.5 chance of Mike and we want a 1, how do I change these weights here to get slightly closer to 1? and I do it. And then I say, "how do I change these again to do even better?" and so on and then I work my way back, ok, that kind of maths we're not gonna go into, right. A lot of these things are, are implemented in libraries So as a researcher, I mean, much as I'd like to implement some of these things, it takes quite a long time just because programming takes a while, right, and And, it's better for me just to apply these things and get good results than it is for me to reinvent the wheel all the time, constantly, if everyone was programming the same things over and over again, no one would get anything done So, I'd have to start by programming up Linux, to get, to get, I'm not claiming I can, by the way, and, and so on. So, you know, let's not reinvent the wheel. Um, so I do this, I send in, let's say 1000 pictures, 500 of which are me you know, so I've been to a photo shoot or something, right and 500 of which are not. And I train it so the convolutions and these weights on the output are such that it gets 1 when it's a picture of me and 0 when it isn't. And then I can look at these convolutions and say "what is it about me that's distinctive?" And it's probably gonna be finding, um, you know, weird shapes on my face, right cause it's a bit of a weird shape, so it - things that are unique to me Now in a more general situation, there's a big database called ImageNet They have a competition every year to see who can classify these images the best. So dogs, cats, planes, trees, and so on OK, they're all in there, and there's a thousand or so images of each right, so, we have a really big network that's much bigger than this little one I drew and we say, "right, let's throw millions of images at this", right, thousands of cats, thousands of dogs and we have lots more outputs than just the one, and we say "what is it?" and it says "it's a dog" and it is. *dog bark* Convolutional neural networks have been around for a little while but, they've really started to be big in about 2012 when it - when someone came along, applied one of these to ImageNet, and got incredible results. And so on and so forth. And now there's this big push and everyone's trying to get even better results, and even better results. Now, I work on more of the applied end of computer science, so I'm more interested in how this affects plant science and things like this So that's what we're working on. Um, but, the kind of results we're seeing are really really impressive So, I mean, case in point, I've done, I've done some root tip detection, so detection of root tips in images, right, of plants and, um, I've got some software that I've already programmed and I've kind of done a low-level feature detector approach to detecting root tips and it's about 70%, ok, which is what you would expect, because maybe some root hair gets confused as to root tip or a bit of blotch of dirt, or maybe there's just two root tips really close together and it gets confused This, the CNN that I trained, um, is 98% accurate And it finds them with 99% accuracy. It doesn't make many mistakes. And that's over thousands of images. [offscreen] So does that mean the work you've done already just goes out the window? Yep. Uh, no, to an extent, yes, and to an extent, no. You need expertise to be able to craft a network and train it and prepare the images. And there's obviously work to be done, and there's some disagreement over how much of a problem you can solve with a convolutional neural network So, there are lots more things you can do with roots beyond finding tips. Can you do all of them with a convolutional neural network? I don't know, we'll see. Are we trying, but, we'll see. Maybe not. So maybe what you do is you use this as a tool, just like other machine learning algorithms, within a package that does lots of other things as well. On the other hand, if you're just doing cat and dog detection, you might as well use a CNN, cause it's gonna do better than anything else. The other purpose for ways the botnet can use its parts is for distributed computing [fades out] [fades in] Now, some objects obviously are more amenable to this than others, but the more images we get, the better it is. There's no depth involved here at all, ok
I think he was kind of tough to understand.
holy shit this is awesome