LSTM next word prediction in Python | LSTM python TensorFlow | LSTM python Keras | LSTM python

I'm sure you are enjoying the advanced NLP playlist guys and we discussed lstm in detail in last video let's go ahead and build a project in lstm and try to understand how this model Works in Python let's do that let's try to build a lstm model in Python okay and the use case we are going to perform is next word prediction use case so the meaning of this is I give a sequence of words okay and my model should be able to tell me what should be the next word in this sequence so for this we will need some training data and we will train our model so that our model is able to predict what should be the next word in that sequence okay first let's understand some Basics about how the training data will be suppose the Corpus or the sentence in front of you is my name is Aman okay so how the training data will be prepared from this entire Corpus is you will prepare features okay you will prepare features and you will prepare a response this is basically a supervised learning problem you will prepare features and you will prepare response suppose the word number my is first word name is second word is three third word and MN is fourth word so how your features will look like is one when when there is my then the prediction should be name okay when there is my and name when both these are together then the prediction should be is when the input is my name is right then the output will be common okay so this is how you will prepare your training data for a supervised learning problem based on your Corpus fine now I am going to show you the Corpus which I will use in Python so as you can see guys this is basically an article of Indian Prime Minister visit to the USA recently you can see here following a lavish State visit by Indian Prime Minister Narendra Modi to Washington U.S President Joe Biden so all these things is basically comes from a news article I will take this article I will train my model and I will try to predict if I give some input sequence then what will be the output fine so as I was telling you if you compare this corpus with my small Corpus my name is Aman right then in this Corpus how the training data will be prepared so training data will be the first word following right the first word following if model sees this word then the next word should be a if the words model C is following a the next word model should predict is lavis if the model C is following a lavis the next word model should predict its state if the model sees all four words the next word model should predict is visit if you have understood this you have understood the concept of how the train data will be prepared remember the length of all these will be different so in first case in this case only one word I am giving as input in second case two words I am giving as input in third case three words I am giving as input okay that does not matter much our models would be intelligent enough to predict the next word given on any length but when we are training we need to keep the track of this by padding the length of this okay so padding is NLP concept which deep learning concept which I have explained many times in my videos you can watch those having said that let's go ahead and try to build this in Python okay so I am importing here packages guys most of these packages I have used and explained you in ordinand videos the link is right here if you have not watched please watch that apart from one new package that is lstm here okay because I will be using lstm model now what I'm doing is I'm importing the same text here same text following a levy State visit by Indian Prime Minister and you can see some you know there are blank lines so spaces are there and new lines are there right so what I will do is I will process this line by line so if you go to my text right this this is one complete sequence the first complete sentence is one complete sequence okay this is one complete sequence So based on that I will process this one by one for example this is one this is another this is another like that okay so let's go to the model and first of all I need to tokenize this right I was telling you here right every time you are working with text Data you need to tokenize it so you need to break the sentence into different words and assign these words a number okay so this is a very first process that you do in any NLP task same process I am doing here if you see I am just breaking it into text by calling fit on text okay and I am just creating a number for all the words for example the word d will be known as the first word two will be known as the second word in will be known as the third word and so on and so forth okay what I'm going to do next is I am going to prepare my sequences sequences means I am going to prepare okay these words are cut together in a sequence so how I am going to do I am going to run a for Loop here if you see I am saying my input sequences is equal to blank then I am saying for line in my text Dot split on new line so I am saying split this entire text on new line and print line I am just printing line just just to make you understand what I'm trying to do in this for Loop okay so when I print line right you can see all these lines are coming one by one this is first line This is second line right and let me let me uncomment this and comment what will be the tokens okay so in this line what are the tokens so for example first line word number 99 4 and 177 just pay attention to these three words guys 99 4 177 okay so if I go to my Corpus 99 should be following a should be 4 and lab is should be 177 so let me see in my text 99 you see the word number four here a 99 should be following okay so if I go here 99 should be following okay and 177 should be lavish 177 should be Levy c177 is levels so what I am doing here is I am just creating a numeric representation of the text that you are seeing in my Corpus okay this is very simple to understand after that what I am doing is for I in range of 1 to length of token list so I will go in all this one by one and I will see how the words are occurring together so let me print it and show you it will be very easy for you to understand and gram sequences my engram sequences I think some mistake here my n gram sequence myanogram sequence name and gram sequence is not defined okay here it will be my engram sequence okay so if I print it like like this then you will be able to see that I am taking all the combinations just for the training purpose so as I was explaining you here right in theory first if what if model sees only if what if somebody enters only first word okay then first and second only that thing is there to learn right what if somebody interests two word then two words and three words this will be one combination right then what if somebody interests three words three words fourth word is the combination to learn right so here what I am trying to do is I am trying to say when word 99 is there then 4 is there now three combination when 99 4 is there then next is 177 1994 177 is next is 50 1994 177.50 is next is 34. 1994 like that right so based on if somebody enters how many words it will predict the next word okay so based on that I am preparing my input sequences okay so my input sequences will be prepared and in the my input sequences right so in my input sequences all those sequences will get stored now what I am going to see is I am going to see max length of my input sequences and I am going to pad sequences now you if you don't understand padding concept guys I have used it many times this is just to make sure that all the length are same for your training purpose okay once I pad this you can see input sequence and let me try to show you the very first sequence here okay see the very first sequence here this is 99 4 this is this is your 1X so suppose this is your first record so what I mean to say say to you is when you see 99 4 right that is my name okay when you see this is 99 4 right and if you see the one right you will see 994177 so this data is basically my name is okay now what what will happen is we will break it into features and targets so as I was telling you the apart from last word right last word will become Target and everything else will become feature so if I have to break for example this one then what will be my feature and what will be my target 99 up to here whatever I'm highlighting in blue will be independent features and 177 will become Target okay if I have to take the first one what is the feature and what is the target whatever I'm highlighting here is the features 4 is the target okay so same thing I am doing here in X everything up to the last one in y only the last one now let's see here guys X is there and Y is there now let's compare again with the Corpus it will be very easy for you to understand so let me take X 0. you see in X 0 what I am saying is this is my input word number 99 is my input and what is my y 0 y 0 is 4 so where from 99 and 4 as 4 are coming go to your Corpus following is your 99 number word a is your phone number word okay I hope this is clear if this is clear then everything is clear okay so one what is my one 99 not input sequences I want to see X1 99 4. 99 4 is my input okay and what is my output in this case it should be 177 so what I am telling to model every time you see following a the prediction is lapis okay So based on that now one more thing we have to do before training the model we have to convert this Y into one hot vectors because that is how model will take it okay so just from tf.2 categorical I am converting y to one hot vectors it's very simple to understand one hot vectors means everything else will become 0 apart from that word which you are trying to represent so for example for the first observation fourth word right should be one everything else would become zero so zero one two three four if you see fourth is one rest everything is zero okay rest everything will be zero I am seeing y zero if I see y one right then 177th number should be one everything else would be zero so if you see here wherever 177th will be there right that will be one rest everything will become zero so just a one hot representation of Y so what we have done till now we have prepared our X and we have prepared our y now let's go ahead and Define our model so I am taking a sequential model I am adding an embedding layer all these things explained in my RNN video nothing new here just one new thing lstm 150 units I am taking so that I should be able to get a good result and I am taking a denser in the output and soft Max because this is a uh this is a scenario of classification right multi-class classification basically so I will take this and I will prepare my model and on 100 Epoch I will train this model okay so let me train this model here on 100 epochs so when the model is training let me explain you what I am doing after this okay after this what I am doing is I am taking something known as input text that is my India for example now I will try to predict okay so input text I am taking India and predict next words I am taking 2 which means after India predict predict next two words okay so what I'm doing in this for Loop is how many words I need to predict right I will take that in a loop and I will just pre-process my token pre-process my token means what is my input word I will pad sequence okay whatever the pre-processing I need to do I am doing here then I am predicting okay and once I predict what I do is I just add that prediction to my input so that next prediction should be better okay so if you see here I'm saying input text is equal to Output word so what will happen is suppose India comes okay India goes into the model for prediction and a new word let's say Aman predicts then in the next iteration what will happen is India among both will go as the input and third world third word will get predicted okay so what I am trying to do in this for Loop is I am trying to enrich the experience of my model so suppose any length you want to predict here I am trying to predict two length suppose I want to predict three length then this four Loop will run three times and every time this input text will get updated with the prediction okay so what will happen is it will make my prediction rich that is what they target here so I will give you this notebook guys you can you can take it from my GitHub and you can do your experimentation all in all what's happening in this for Loop is it is taking the input word it is predicting the next word and sending it back to the model for the next word prediction and finally when all three WhatsApp will be predicted it will give you the result okay so I will pause this video for some time because as you can see I have run 400 epochs 20 books are running okay it may take few minutes I will take a pause here and I will come back when 100 epochs are completed so guys the model is trained now and I am going to give a input text for example here India and I am going to tell to the model predict next two words okay so when I run this model as I was telling you I am printing knowingly here the tokens so that it's easy for you to understand so what will happen is in first round six six is token number India okay that will go as the input and the prediction will be five number word okay so six five in the next iteration six five both will go as the input and the prediction final prediction will be India and D okay so suppose I make it three words I want to predict three words and I am giving India as the input so let's see what is the what is output here so India and the US is the output for next three words so if I want to change this for something let's say this okay US president name and I want to predict next let's start with two words okay so if this job done is the input then what is the next two words this is the two words okay let me make it three words so you will see here some meaningful information will come Joe Biden and Indian Prime let me make it five words so what makes sense here is Joe Biden and Indian Prime Minister maybe I'm assuming that might be the output okay so as you can see here the output is Joe Biden as Indian Prime Minister Narendra after this the next word should be Modi Modi I believe because that is the name right so if you see here that is exactly what what I was saying you right so this model is doing good based on the input that we have given the input is very small right this Corpus is very small this is not a large Corpus so this input we have given and what we are doing here is every time our model is predicting a new word right we are feeding that word back to the prediction mechanism and telling take help from this and then predict the next word okay so as you can see this thing is happening here and it is performing decent so just to revise what what all we did guys very simple things we did we imported our Corpus we we did a tokenization assigning numbers and then we broke it into sequences we took the sequences we did the padding because that is a normal thing we do in on NLP task and then we we broke the features and Target that I put lot of focus for you to understand how to break it into features and Target once you do that it's quite simple we are just adding lstm layer and making our prediction okay so I'll give you this notebook guys take it uh play around with it do some improvements on this and comment me like how did you like it and please give me a thumbs up if you like this video I'll see you all in the next video guys wherever you are stay safe and take care

Channel: Unfold Data Science

Views: 10,673

Keywords: LSTM python tensorflow, LSTM python keras, LSTM python example, LSTM python code, LSTM python code github, LSTM python sklearn, LSTM python code from scratch, LSTM python implementation, LSTM python code for time series, LSTM python model, LSTM python example code, LSTM python keras example, lstm model for time series forecasting, lstm stock price prediction, long short term memory, unfold data science deep learning, advance nlp, unfold data science nlp

Id: jTjLf0ytSw8

Length: 17min 31sec (1051 seconds)

Published: Mon Jul 24 2023