Overfitting in a Neural Network explained

[Music] in this video we'll discuss what it means when a model is said to be overfitting we'll also cover some techniques we can use to try to reduce overfitting when it happens we briefly mentioned the concept of overfitting in a previous video where we discussed the purpose of a validation set let's build more on this concept now overfitting occurs when our model becomes really good at being able to classify or predict on data that was included in the training set but is not so good at classifying data that it wasn't trained on so essentially the model has over fit the data in the training set so how do we know if our models overfitting we can tell that the models overfitting based on metrics that are given for our training and validation data during the training process we previously saw that when we specify a validation set during training we get metrics for the validation accuracy and loss as well as the training accuracy and loss if the validation metrics are considerably worse than the training metrics then that's indication that our model is overfitting we can also get an idea that our model is overfitting if during training the models metrics were good but when we use the model to predict on test data it's not accurately classifying the data in the test set the concept of overfitting boils down to the fact that the model is unable to generalize well meaning it's learned the features of the training set extremely well but if we give the model any data that slightly deviates from the exact data used during training it's unable to generalize and accurately predict the output overfitting is an incredibly common issue so how can we reduce it the easiest thing we can do as long as we have access to it is to add more data the more data we have to train our model on the more it will be able to learn from the training set also with more data we're hoping to be adding more diversity to the training set as well for example if we train a model to classify whether an image is an image of a dog or cat and the model has only seen images of larger dogs like labs golden retrievers and boxers that in practice if it sees a Pomeranian it might not do so well at recognizing that a Pomeranian is a dog if we add more data to the training set to encompass more breeds then our training data will become more die and the model will be less likely to overfit another technique we can deploy to reduce overfitting is to use data augmentation this is the process of creating additional augmented data by reasonably modifying the data in our training set for image data for example we can do these modifications by cropping rotating flipping or zooming we'll cover more on the concept of data augmentation in a later video but for now I'll pop up a video on-screen regarding data augmentation for my Kerris playlist the general idea of data augmentation allows us to add more data to our training set that's similar to the data that we already have but it's just reasonably modified to some degree so that it's not the exact same for example if most of our dog images were dogs facing to the left then it would be a reasonable modification to add augmented flipped images so that our training set would also have dogs that face to the right now something else that we can do to reduce overfitting is to reduce the complexity of our model we could reduce a complexity by making simple changes like removing some layers from the model or reducing the number of neurons in the layers this may help our model generalize better to data it hasn't seen before now the last step I'll mention for reducing overfitting is something called dropout the general idea behind dropout is that if you add it to a model it will randomly ignore some subset of nodes in a given layer during training ie it drops out the nodes from the layer hence the name dropout this will prevent these dropped out nodes from participating in producing a prediction on the data this technique may also help our model to generalize better to data it hasn't seen before we'll cover the full concept of drop out as a regularization technique in another video and there we'll understand why this makes sense so hopefully now we understand the concept of overfitting and why it happens and also how we can reduce it if we see it happening in one of our models and I hope you found this video helpful if you did please like this video subscribe suggest and comment and thanks for watching [Music]

Channel: deeplizard

Views: 82,757

Keywords: Keras, deep learning, machine learning, artificial neural network, neural network, neural net, transfer learning, AI, artificial intelligence, Theano, Tensorflow, CNTK, tutorial, cuDNN, GPU, Python, supervised learning, unsupervised learning, Sequential model, image classification, convolutional neural network, CNN, categorical crossentropy, relu, activation function, predictions, stochastic gradient descent, educational, education, fine-tune, data augmentation

Id: DEMmkFC6IGM

Length: 4min 15sec (255 seconds)

Published: Wed Nov 22 2017