Convolutional Neural Network in Matlab

in this lesson we will learn about convolutional neural network in short continent it is actually a deep neural network that is specialized for image recognition Continent works in the same way our visual cortex works and recognizes images it may seem little complicated at the beginning but don't worry I'm here to help let's start with the basic architecture of continent we can divide the entire architecture into two broad section featured learning and classification section the input image enters into the feature extraction Network the extracted feature signals enter the classification neural network the classification neural network that operates based on the features of the image and generates output the feature extraction neural network consists a pile of the convolutional layer and pulling layer pairs the convolutional layer as its name implies converge the image using the convolution operation it can be thought of as a collection of digital filters the pulling layer combines the neighboring pixels into a single pixel therefore the pulling layer reduces the dimension of the image as the primary concern of content is the image the operations are the convolution and pulling layers are conceptually in a two-dimensional plane this is one of the differences between continent and other neural networks now we will learn about the convolutional layer convolutional layer generates feature Maps from images the working principle of this layer is different from other neural network layers it does not employ connection weights and weighted sum instead it contains filters that convert images we call these filters convolutional filters the number of feature map and the number of convolutional filter is same that means if there is for convolutional layer then it will generate for feature maps the filters of convolution layer are two dimensional matrices to understand how convolution works let's take a 4 into 4 pixel image and to convolution filters I'm taking these filters arbitrarily but in actual continent these filters are determined through training process let's start with the first filter the convolution operation begins at the upper left corner of the sub-matrix that is the same size as the convolution filter let's apply the convolution this one will be multiplied with this one the 0 will be multiplied with this one the 0 will be multiplied with this 4 and this one will be multiplied with this 6 the result of multiplications will be added together and the value of that addition will replace this entire block the result of the convolution will be like this this 2 into 2 block will be represented by a single value in this case that is 7 the next convolution operation will be performed on this block this one will be multiplied with this 1 this 0 will be multiplied with this one this 0 will be multiplied with 6 and 1 will be multiplied with 4 finally the results of these multiplications will be added together which will represent this entire block in our case it is 5 here is the result of these convolution this 2 into 2 block has been replaced by 5 using the same calculation the next block will be replaced by nine after that this block will be replaced by four this process will keep going and the final result will be this one here at the final stage before in 2 4 pixel image has been converted into a 3 into 3 pixel image the largest value at the final result is 32 and it was generated from this block have you noticed the similarity between this block and the convolutional filter they are similar both of them have value at 1st and 4th position and no value at 2nd and 3rd position the convolution operation yields large values when the input matches the filter on the other hand when we performed convolution operation on this block it yielded for although it has the same significant number Totti but it doesn't create that large value it happened because of using mismatching filter if we use our second convolution filter then this will be the result notice carefully here this block generated 36 that is a large value it happened because the filter we are using now matches with the input block this example demonstrates that the feature map will be extracted depends on the convolution filter you are using have you noticed there is another layer in between convolution filter and feature map these are activation functions here is nothing special about activation function in component they are same as the activation function we used in neural network a rectified linear unit in short really is the most popular activation function however we also used sigmoid function and 10h function when we need them we have covered convolution layer let's talk about pulling layers now the task of the pulling there is to reduce the size of the image the operation of the polling layer is very simple and straightforward to understand how it works let's take the same image we have taken earlier to perform convolution operation there are two types of pooling max pooling and min pulling for a mathematical perspective calculating the mean polling and Max pooling is convolution operation the difference between the convolution layer and this convolution operation is a stationary convolution filter and the convolution area do not overlap therefore if we take 2 into 2 matrix as the convolution area there are 4 convolution areas for the mean pulling the calculation are done by taking the means of the convolution areas for example the mean of the first area is 1 plus 1 plus 4 plus 6 over 4 that yields tree the mean value of the next convolution area is 1 plus 3 plus 4 plus 8 over 4 which means 4 in the same way the mean value of next to convolution area are 8 and 3 respectively this is how the mean pooling values are generated calculating the value of max pooling matrix is very easy simply take the largest value of each convolution area and form the matrix the highest value of the first convolution area is 6 second convolution area 8 third convolution area 30 and fourth convolution area 5 the max pooling matrix is generated using these values these are the theoretical concept that we need to learn before implementing a convolutional neural network as we have completed the theoretical part let's proceed to the practical section in this section we will implement a convolutional neural network that can recognize handwritten digits we will implement the neural network using M list database it contains image of 70,000 handwritten digits 60,000 for training and 10,000 for validation each of these images are 28 into 28 pixel binary image like this picture however training a neural network using 60,000 image is time-consuming so I am showing the process using 10,000 images where 8,000 for training and 2004 validation the digits can be classified into 10 classes 0 to 9 a 28 into 28 pixel image has 784 pixels therefore there will be 784 input nodes we are taking a single convolution layer with 29 into 9 convolution filters output of the convolution filters will be passed through a rayleigh function the pulling layer has 2 into 2 sub matrices our classification neural network will have a single hidden layer and output layer it will have 100 nodes and the activation function will be a real new function as we have to identify 10 classes we will construct the output layer with 10 nodes and we will use softmax function as the activation function of output nodes this is the architecture of the neural network we are going to implement there are many layers here but three of these contains weight matrices so these three layers require the training here w1 is weight of convolution layers it is used by convolution filters for image sesang w5 and WL contain the connection weights of classification neural network there are input nodes between pulling layer and hidden layer this layer transforms to the image into a victor it is not involved in any training operation so we have represented it using squires let's implement the system click on the browse for folder icon to specify allocation create a folder there to save your program here I'm creating a folder named implementation once you have created the folder get inside the folder and click on select folder button our first task is to load the M nest images to do that we are going to implement a function let's call it load em nest images declare the name of the function input type and return time this function will take M nest image database as input trial and will return image data we're using if open function to open the file the file is a non text type so we have to use our V as an argument you this line will show an error message if it cannot open the file in the next line we are reading the data from the file we have opened earlier and loading the data into our variable named magic if there is an error during reading the data the next line will prompt an error message the next three lines read the number of images number of rows and number of columns and store this information into three variables named nun images nun Rose noncoms respectively in the next three lines we are reading the image reshaping them using the number of columns rows and number of images and ordering them accordingly at the end this function returns reshaped 8-bit images now let's create a function to load the label I'm naming it as load amidst levels declared the name of the function it takes file name as input and returns the level this line opens the level file and if there is any error while opening this line prompt an error message this line reads the level from the file and store it in a variable named magic and if there is any error while reading those levels into the variable it prompts an error message using this line this line reads the number of levels and the levels are stored in the variable named levels in unsigned character format using this line if there is any mismatching while reading the level this line will prompt an error message so we have implemented two functions to load images and levels from em nest database let's implement the activation function now we will use two activation functions value activation function and softmax activation function let's start with creating real new activation function so this is the really activation function let's implement the sub max function now this is the softmax function so we have implemented the really function and the softmax function we have to perform convolution operation so let's implement a convolution function this function takes two input X and W and we're not implementing the convolution function in cold MATLAB has already a built-in function to perform convolution called calm too we are simply calling that function to perform convolution operation let's create a function to implement the cooling layer am naming the function as pool you the pool function takes one input and it used matlab's built-in comp to function to perform convolution let's implement the function to train our convolutional neural network the name of the function is M NIST Kampf you it is a pretty lengthy program and beyond the scope of explaining each and every line if you don't understand a part of this program please comment below I will try my best to explain as elaborately as I can to test our content we need to implement a random function that can generate random array and random number this function will serve the purpose to train the network we need to download m nist data set we will train the network using 10,000 image so download this data set you have to download the levels as well then create a folder in the directory you are working on naming it inist and extract the data set in this folder you can remove the GOP version after extracting them now you will see the data set in your current directory and now we are ready to test our program to test our program I've created a function named test M missed cond you this function calls the amnesty function and train the convolutional neural network and also show the accuracy of our trained network this is the first epoch and this is the second a park and the final one and the accuracy of our continent is 94% let's implement a function to visualize the performance of our convolutional neural network you now we will create a script named cloud features this script will use this play Network function and will plot the features extracted by our count that you let's run the program this is the result of our program let me organize it a bit so that you can understand it properly you this is our input image these are the convolution filters this is the result of the convolution and when the result of the convolution was passed to really function that is our activation function this result was generated finally when these features were processed through meme pool we found this final result this is the end of this lesson I hope this lecture helped you to understand the concept of convolutional neural network and he were able to implement the code in MATLAB if you have any question regarding to this lecture write down your question in the comment section I will try my best to answer it you

Channel: Nuruzzaman Faruqui

Views: 81,806

Rating: 4.9210525 out of 5

Keywords: Convolutional Neural Network in Matlab, ConvNet, convnet, cnn, CNN, convolutional neural network in matlab, training a convolutional neural network in Matlab, Practical ConvNet, Convolutional neural network in matlab example, Convolutional Layer, Pooling layer, Mean Pooling, Max Pooling, Activation Function, ReLU function, Softmax function, extracting feature map in Matlab, Feature map

Id: ZOXOwYUVCqw

Length: 26min 11sec (1571 seconds)

Published: Mon Sep 17 2018