Node JS - Readable Streams

what's going on guys welcome back for a brand new video wait for to a brand new video this is web dev journey and in today's video we're going to be talking about readable streams we are going to get into the other streams writable streams you know duplex transform streams but right now in this video let's take it slow let's take it slow this is probably a topic that everybody knows or probably doesn't know so we're gonna be talking about readable streams now if you did not know a readable stream uh reads data from a source and then feeds it into a pipeline bit by bit for instance we can create a readable stream that can read from a source array like this one right here that we have right here advice is take a moment and just you know this is golden advice all right so take it so what we're going to do is we're going to create a particular type of stream that can read the data from the array and then pass it along chunk by chunk so let's say what so let's see what would what that would look like okay i'm going to go ahead and create a new instance don't worry a new instance of this uh advice stream so i'm say cons cons of advice advice stream let me just say stream right and we're going to create a new instance of this using the class stream from array but we don't have this just yeso stream stream from array we don't have this just yet we are going to be building this class but right now we're going to what we're going to be passing in this uh class is the advices okay advice this there we go now streams also trigger events and some events that we're going to be looking at are data and and so basically the data event once the data event is triggered that that just means that a little bit of data is passed back to the callback function and let's try that right now so we're gonna say uh from the advice stream right we're gonna call on so on the the data event data there it is like i said there's going to be something passed back to the callback function and we're going to call this chunk right this is a chunk of data that's being passed back so for right now what we're going to do is just log out console.log and then chunk so that way we can see what exactly that what what does this look like right what does this chunk look like chunk don't know why there's a semicolon right there now another event stream that we could talk about is the on end we if you want to figure out when the uh when the uh stream ends you call on this event and well this is kind of self-explanatory right it it ends that it ends right so you know it ended so you want something to do once the stream ended you want to do something else for right now we're going to just log out what log log log okay we want to just log out you know um done you know something like that just like so okay now since we are creating a stream we are we're going to be using a module called stream and we need to grab what happened what happened we need to grab the readable stream from that module so const or sorry sorry something we're going to grab that from require and this is already a pre-built a pre-bit module and it's called stream there it is and i think can i grab it now please tell me there it is readable that's all we want this is a this is the module that we need the readable stream module and we're grabbing it from the stream module okay now down below this i'm going to be creating that class so or not not class not const class uh stream where's it stream i'm just obvious so i don't make a mistake stream from array we're going to create this class and we're going to extend the uh readable right we're going to extend it's going to extend readable so readable we all know i can't type so this is not a supply surprise that i'm struggling working on it so whenever we extend the readable stream uh yeah whenever we extend the readable extreme we want to implement a read function so what i'm going to do is do read right we're going to implement a read function so the read function is what happens when this stream reads a chunk of data so within this function we want to read one line of the array and then pass it pass it on correct so in order to do that we're going to need to create a constructor so right above we're going to say constructor right we're going to call super no super just like that and what we want to do is just set up some data so this dot array is going to be is going to equal the array that we passed in god damn it so we are like i said we are passing something in which is the advices so we're passing in this array and then we're going to also set the this dot index to zero so that way it will start at zero so like i said this read function is going to take it's going to uh do everything that it needs to you know read bit by bit by bit so in this one we're going to do whoa this dot push we want to push a chunk into the array correct we want to push a chunk of data to the to the array but in order to push a chunk we first need to read that piece of uh chunk or data from the array oh man i really hope i'm making sense so what i'm gonna do is say const chunk we're going to grab this going to do equals this dot array and we're going to be grabbing the very the the index whatever this dot index is so in this case it's zero so we're going to grab the very first index of this array and then push it to push this out so don't don't get this confused this dot push actually pushes this chunk of data into the stream not to an array not to anything else but to the stream right so don't get that confused to anything else that's why we said this stop push this refers to the readable stream that we're uh that we're creating right this dot push chunk we're pushing this chunk to the stream all right so anyways and then what i want to do is just say this dot index we're going to increment it by one no not by zero by one but since we are creating our own stream we do need to figure out hey when when did the stream end or when did this advices end right so that's pretty simple we're going to say if statement so if this dot index is less than or great or greater than this dot array dot length there you go then that means that this has ended and we should no longer read so whatever it do there so if that that's good else meaning that it's all done what we're going to do is this dot push no since we are pushing gnaw or nothing you could push no or nothing into the stream but since we push nothing or not to the stream that this is going to indicate that hey we're done there's nothing else to be reading and this is going to end the stream okay now if i control save this and if we try this out i could do node node i do it's called the readable.js if i do that you're going to see that hey look at this we are pushing every single little thing this is a console log right here chunk and remember the chunk is the little bit of data that it sends in right so you can see that we're reading our array bit by bit and all of these little buffers are the chunks that means that the stream is operating in binary mode so it's reading each line of the array and each chunk is showing us the binary data now streams have two two modes okay binary mode and object mode and when they're in binary mode we could actually read the data as binary or we could read it as a string so in order to read this data as a string all we need to do is set the encoding type to you utf-8 utf-8 and we could do that or where am i we could do that in the constructor right well the super right here the super we're going to pass in an object we're going to say hey we want the in encoding to be utf-8 even tells you right there right or utf-8 although you could do it the other way but i'm gonna do it like this just make double share i think you could do it the other way as well and if i do again no dot readable you're gonna see that hey here is our uh what's it called our advices now like i said there is another type of mode and that's the object mode so we can pass pass binary data through restream we could also pass string data by sending the encoding type and i can also specify whether the object mode of our readable stream is on so let's actually create a no well let's actually convert this into a readable i mean to readable to an object right let's do this object mode so let me get rid of this let's go over here and we do need to say we we do need to tell the reading i'm gonna just copy this we do need to tell the stream hey this is going to be running on object mode so i'm going to say object mode we're going to set this to true it takes in the boolean so true and i want you to put this on the side so that way you know that encoding converts buffer to string so that way when you get this from github you know what this is right so right now we're saying that hey we want the we want this stream to be in object mode meaning that it's going to be giving us an object instead of just strings or binary right and with that we do need to say when we're passing in this chunk right we're pushing this to the stream which is the array which is not correct now what we want to do is pass in an object and inside that object we're going to have data that's going to be set to this dot array um and then this oh my god this dot index right we're giving it whatever the index is and then we're going to say index my mouse will just pass in the index right do this dot index as well and don't do not do a semicolon right there there you go so let's try this out ctrl save let's go over here let's clear this out and you're gonna see that bam check that out we got uh our first one index zero one two three and taken notice that it does give you a very the very last one it gives you data undefined in index five since we are saying that hey uh since uh remember this is the last thing that it will push no so just remember that it is going to give you an undefined or something like that when you end the stream if you're in object mode so you might be wondering why are we creating a stream if you know node.js already provides one for us right that from the fs module that i'm pretty sure a lot of people already know about right and this was solely for the reason so that way you know exactly what's going on in the back of like streams right like what goes into it or this is it goes this is obviously there's more to this but this is just like the basic of basics of it so anyways let's actually start uh creating creating a stream from the fs module that we're also familiar with right so i'm just rename this to readable copy that i'm gonna say readable class so that way you know this is the class that we made i'm gonna create a new file and we're going to call this readable fs so that way you know that this is for the fs module so in here we're going to do a const f fs we're going to oh look at that did it do that what if fs require oh look at that that's cool fs but we do need to put that oh god do it there you go fs and this should really be single quotes well it doesn't really matter i just like it to be single anyways anyways so we're grabbing the fs module and what we're going to be doing is creating so const a readable stream so i'm gonna say const read stream and we're going to create a new instance of this from the fs dot create read stream and what we're going to be reading the you could pass in an option of like what you're actually reading so what we're reading is this mp4 that we have right here so um there you go that's what we're going to be reading obviously you could pass whatever you want in here it could even be a text file if you want it to be but right now i haven't end before so we're going to be reading this mp4 now let's do the exact same thing that we did over over here in the class is with not with the class uh get this listen for those events the data that end and there's one more that we really should be listening to and that's the error uh event handler so we're going to do read stream dot on um on data which is going to send some no not what dancing data it's going to send some chunks some chunk to the callback function i'm going to just log out simply log out log out little chunk slash n so that way we go to the new line and then we're going to spill out the the actual chunk there it is so we got this one i'm copy this we also have the on end which it doesn't really pass anything and this is going to say read stream and then there you go and we also have one more and this is going to be this is one that i didn't want the one that i didn't talk about but there is a error event so if it never does occur what we could do is just cancel log and error has a curd and we could also get a console console dot error there you go and then pass in the error as well and there you go so if i control save this let's actually see this in action clear this out now this is going to be very very long this this this output is going to be very long why do what hold on oh uh i did the wrong i did i tried to do node so node readable class not not class fs there you go you can see that you see that it's very very long because this is the mp4 so anyways you're going to see that we have this buffer chunk this should not be new to you anymore we do have a little chunk by chunk by chunk by chunk little chunk little chunk a little chunk right and you can see at the very end it's going to say it should say and always got red restream ended now this is just for fun if you want to see how how what the size of this chunk is you could literally do chunk.link there you go i'm going to say size right here so size and then ctrl save this clear this out and then what i'm going to do is readable fs and you can see that here it is the size and it's giving you the size and you can see that it's reading it from finish i mean from start to end right and the very end one is a very little size now node.js has streams everywhere now if you did not know standard input or stud in as people know it is a readable stream so let's actually implement that so over here we could do a process dot stud in and there it is stud in and then since it is a readable stream we could do on and then listen for that data or listen for that event data get a chunk back chunk back and then you know don't worry if if this if uh if we don't have it set to uh what's it called utf-8 we could obviously just to string it and then it'll do that for us and it'll convert it into string so what i'm gonna do well not i'm gonna say let let text we're gonna set that equal to the chunk that we got back dot to string there you go and then dot trim just for good measure trim there you go and then what we're going to do is just log out that bit of text so text and i'm not actually going to come this out so that way we don't see this ctrl save and let's give this a shot so if i go back over here clear this out let's try to do node and you can see that restream ended but our process is still on so if i type in well hello you're going to see hello back and just to make this a little bit more clear i'm going to just say echo right here echo and that's supposed to be a c right there so that way you know exactly what's going on clear this out let's try it again okay so if i say hello so it's this stud end is waiting for an input to be a standard input which is our keyboard is waiting for an input to uh come in so that way it could read that piece of chunk or piece of data so this one we're at we need to ask for the data so this stream is a non-flowing mode okay this kind of stream where you have to ask for data is non-flowing mode the stream that we have over here where it just reads it right off the bat is called flowing mode so we could actually enter any stream into non-flowing mode or flowing mode so let's make this restream of our mp4 into a non-flowing mode so the way we're going to do that let me get rid of that is instead of doing this what we're going to be doing is we're going to make the make this restream pause right off the bat is going to be on pause which is going to you know not read it's not going to be reading right anything so in order to read data from the read stream now what we want to do is um say right here well if we hit enter or any key by by that matter of fact we're going to start the read dot stream or read stream dot read we're going to start this process once again but like i said read there you go we're going to read the stream dude come on and it's going to be reading the i should say that something else but restream restream we're going to be reading the restream it's going to be reading a chunk by chunk by chuck bit by bit by bit right so let's unconsole this control save and let's see how this works right so right now off the bat is going to be paused so if i go over here clear or ctrl c and then clear this out let's try to do that one more time you're going to see that nothing appears right but if i hit enter you're gonna see that hey we're reading a bit of chunk right here if i hit enter again there you go hit enter hit enter you enter now we made this into a non-flowing mode stream okay we end this stream into a non-flowing mode now to make it back into a flowing mode all i'm going to do is say that hey if you type in the word finish let's go let's go and just go into the flowing mode which is going to just read all the bits without without pause or anything right so right here i'm gonna say if i'm gonna say the chunk dot to string dot trim so if this equals finish there you go if this equals finish what we want to do is read stream read stream dot resume which is going to enter back into flowing mode which is just going to like spill out all of the rest of the uh mp4 whatever that is right so let's control save this and let's let's give this a shot let's go clear this out and there you go if i enter it's going to give us a bit it's going to give us a bit bit bit bit but if i type in finish it's going to go into flow mode and there you go it's doing everything on its own which is really really good right that's pretty cool so as you can see readable streams are everywhere learning to work with readable stream is an essential skill for a node.js developer and this is what this series is going to be about streams man trying to give you trying to get you guys comfortable with streams so i hope this video helped you with at least the readable side of the streams now we're going to get into the writable stride of strings in the next video um please hit that like button if you learned something uh like like the video well they're gonna say hit the like button like the video well you know do both like it well you can't right it'll unlike it i don't yeah i don't unlike it so just hit once hit that like button once comment down below and subscribe to my channel if you have not already subscribe to it man i've noticed that there's a lot i think is 80 of viewers are unsubscribed that view my videos so if you're not subscribed and you're enjoying my content please subscribe hit that bell notification i've never said that before it feels weird saying that and thank you for all you guys that have been supporting me i really do appreciate it i appreciate you spending your time you know spending time out of your day to watch my videos it means a lot to me so thank you guys and i will see you in the next video peace

Channel: Web Dev Journey

Views: 1,482

Keywords: node js, nodejs, nodejs streams, node js streams, nodejs buffer, node js buffer, readable streams, node js readable streams

Id: _pqv06ySvuk

Length: 23min 0sec (1380 seconds)

Published: Mon Dec 21 2020