Learn Object Detection in the Browser using TensorFlow.js | Computer Vision Web Development Course

in this video tutorial we will be creating a web app to do object detection on images and on live webcam feed using tensor block JS so here is the quick demo of how our application works so if you use click over here you will be able to upload an image so you can just upload any image okay so here is the RGB image which we have uploaded and here is the gray scale image and if I click over here we will be able to detect the objects uh different objects in this image like you can see over here we are able to detect the bus traffic light person and along with this we also have the confidence score here as well so in the similar way you can upload any image and you will be able to detect the objects inside that image and if now if you want to do uh the object detection on the live webcam feed you can simply click over here and if you just go to inspect from here so here you can can find all details as well like model loaded and over here if I just click on enable webcam so this will enable the live webcam feed and you can see over here uh it is able to reect me as a person over here okay and if I just pass on the cup over here let's see so now you can see it is able to re the cup as well and if I just uh ask my mobile phone over here so now you can see that it is able to like the cell phone over here as well okay and anything else again C so now you can see the reduction results are very good and we have created this conrete web app using HTML CSS and JavaScript and if you want to create this similar app let's explore this in this tutorial in the first step we will do the complete web layout design so over here I will just go over here and create a new folder and write right over here section 07 object detection okay so inside this folder I will create three files index.html style. CSS and the third one will be main.js so I have Creed Three different files over here index.html style.css and main.js files okay so over here you can see this files so as we go ahead first of all I will add the boiler plate code so I will uh press shift exclamation mark and click on enter over here you can see the boil trate code and I will just change the title of our web app and I will just write over here object detection okay so I just change the title of the web app and over here I will link my CSS file so I will here write link to the link to the CSS file so I just write over here link and just CSS and here you can see the name of our CSS file and in the end of the body I will link the Javascript file to the JS file and I will just write a script source and over here I will just write the name of the JavaScript file okay so we have linked our uh CSS and JavaScript file so let me show you what kind kind of web layout we will be designing so over here uh this is the just you can see that this kind of layout we will be designing you can see the name of the application as Photoshop but the name of our application will be object detection application okay and over here you can see that this is basically a main container and we can divide our main container into three different containers so we will have a main container and we will further split our main container into three different containers so like you can see over here so we can split our we can uh consider this lay complete layout as a one container and we can split that container into three different containers so this will be our first container this will be our second container and this will be our third container so over here you can see that this is first this is a second one and this is the third container okay so if I just uh give you a more General overview this the first will represent our side like the second will represent our sidebar and this third will represent our right panel so first will be the side side line the second will be the sidebar and third will be the right panel okay so then after creating uh three different containers sideline sidebar and right panel then we will add some icons in the sidebar and after adding the icons in the sidebar then over here in the right panel we will have the header and this will be our tablet area so this is how things look like so this is a complete layout of our application how our application looks like then we will split our application uh uh we so this you can say that this is complete one container we will split into further three different containers which is the sideline sidebar and right panel in the sidebar we will add further than icons different icons like uh upload an image then this represent the image RGB image this represent the gr scale image icon so you can see that we have different icons and uh whenever you click an icon different functionality will be performed so after adding different icons then we will create a right panel this is the right panel we will split our right panel into header and the tablet area Okay so this is how our layout looks like so let's start writing the code so we can simply close this so we will have a main container so container is nothing it's uh basically a d de so we will have a you can just write over here the main page okay so now we will split into sideline sidebar and right panel okay so I will just write over here main page so over here we will have the sideline so I just write over here D do sideline okay so then we have the sidebar so I will just write over here D do sidebar okay so over here you can de side side bar then we'll have the right panel so I will just write over here right panel and over here I will just write d dot right d do right panel okay so we have created sideline sidebar and right panel so if I just uh show you how our web app looks like so I'm just open this so with here you cannot see anything like you can I don't see any difference like you can see over here the title object detection but can you can't see the side line sidebar and St panel so for this we need to write our CSS code so let's jump into it so I will just try so here so I I will just write over here AB section so whatever I write over here uh so whatever I write over here uh this will affect the complete base so I will just write margin as 0 pixel okay then I will just write over here body and let me just write over here background color that's right okay so now you can see over here uh this is how web looks like and now you can see that the complete background color of our application has changed to right okay so we don't want this so I will just write white over here okay so now you can see the White and the red color has disappeared so over here now I will just write over here do main Dash page it so as I told you that we have a a main container and within that container main container we have three different containers sideline sidebar and right panel okay so I will just write over here background color as white again okay and I will just create over here sideline okay and over here first of all I will Define my width as 100% And the height will be okay okay height will be the 100% of our viewport height and I will just write WID as 2% okay so this will look fine and the background color will be uh we can have uh light like so now you can see over here we have put a side like the width of the sideline is 2% so how you this is basically 2% means so if you just see over here if I just change this so now you can see that the S line is decreasing as it changed up uh size of our uh browser okay so but if you just uh write over here if you just write over here 10 PX okay so what this will affect is so now you can see the S side line over here it's very short but if I just uh change this size so now you can see it remains constant it doesn't change okay if I just uh zoom in zoom out our browser so we can just make it uh 2% this look good then we can create a sidebar over here okay so over here I will just write width as 15% okay and the height is will be 100% of the viewport height and the background color will be so for this I will just add uh x value which I have calculated okay so now let me show you so it's very less visible so uh we can have light GRE okay so you can see that okay so now you can see it's not correct so what we can do over here is I will just write over here display Flags so the so as our so main page is basically our parent element okay so when the uh parent element has the property flag so it will affect the sideline sidebar and right panels okay so now you can see that we can see the sidebar okay so now I will just write over here right panel over here and the width will be the reing 100% okay so and the height will be over here will be 100% of The View Board height okay and the background color will be light sky blue light SE green that will also look good so now you can see that we have the sideline sidebar and the right panel so everything looks very good okay so what we can do over here is I can just change this color to my desired color so this this one so now you can see that I changed this color plus uh we will make the color of the right panel white but we'll do it later on so next what we can do is we can add different icons over here in the sidebar so what to add different icons what we can do over here is I will just go over here developers. google.com Das icons okay so over here okay just misspelling it sometime okay so this is the material icons guide I will just write over here fonts. Google d.com d icons and let's open this up from here okay so this is what I was expecting and let's go to the icons over here and over here you can see that uh we have different icons so to add any of these icons uh what you can do over here is you can go to the material icons guide okay and from here you will see the how to get started so here you can see that you set a p how you can use this uh Google uh icon so you can simply copy this from here and you can just go over to your board okay and over here you can simply write add material icons over here and just copy this L from here okay so that's look fine and over here you can just uh catch up all the details how you can add any any icon so for this we will using span and um we will be using icons so we can use any of the icons from here you will just need to write down the name of that icon so let's do this so in the sidebar I will add all these icons so I will just write span and over here I will just write the class name Circle and over here for Icon I will use I I and here I will just write material Dash icons and and the first icon will be the parer so I have already searched for the icons which I want so you just need to write the name of those icons and then I will just write span do Circle and I do material Das icons and the name of the icon over here will be uh contrast okay so let me show you how it looks like so if I just go over here so you can see that we have added two icons over here so let's add the remaining icons as well so for this you can just search over here any icon which you want so first will be our rgp image then we have the gr scale image then we need to uh do object detection on images so what icon we can add over here is okay so we can use this full screen icon on as well so I will just write over here span do Circle I do M Dash icons uh what it is fullen okay so we have added this icon and next what we can do is uh what we can have here is we here so here I can decide video camera so we can use this video camera front as well we can use this icon as well so I will just write span do Circle I do material Dash icons okay and we can just write video camera front so I just write video camera okay so I have added this icon over here as well so you can see over here we have added all these icons which can be seen over here so if you want to like you can see these are not in the correct order so what we can do over here is uh we can de write display Lex we will just write Flex Das Direction column okay and what we will do now is uh you can see over here now there in the verticle so now next what we can do over here is I will just write justify Das content Center and over here I will just write align Das item Center as well okay so now you can see these are know in the center but uh what next we can do over here is uh we can just add justify just as space between okay and let's see so now you can see we have enough space but the icon size is very small so to solve this issue what we can do over here is uh we can just add over here style for das size is 36 pixel and next what we can uh just change the background color to Y okay so you can just copy this line from here and this L this line over here over here over here as well so this looks pretty cool now so okay but where are other icons they have disappear what is the isue okay that this what is show so now it will so now you can see that you can see the icons but the background color should be white okay so let's me close this up first so if I just go over here uh instead of background we can simply add color so you can just this okay so let's look at okay so uh look bit OD now so now it's look fine but one thing we can do over here is uh we can just add over here padding dasht and the beding top will be uh 20% and the pedding dash bottom will also be 20 person okay so okay so to solve this issue what you can do is uh you can just go over here and you can just write box Des sing is equal to border dashboards so now you can see that this issue is very much solved uh so we have this we have this okay so this is a look very much high so we can see the right over here five person and let's so now you can see that look pretty good as well so next thing what we can do over here is uh we can add let's add some comments over here as well main page s side line Side bar right patter okay so next thing what we can do is uh we have added sidebar over here at as well so we can uh so now you can I told you over here that we have a class Circle for this icon so we can just write down the code for this as well Circle so I just write do Circle over here and over here I will just write background Dash color will be the gray color and over here the height will be um 70 PX and the width will will be uh 70 PX as well and what we can do further is we can just write border Dash radius will be 50% okay so now they are not at the center so we can just write display flx FX D direction or simply write over here just justy Das content Center Aline Das item Center okay so now it looks pretty much good okay so now they are very much centered and looks fine so what we can do further now is we can just change this to color okay and this so this looks now very much well so now you can see we have very different icons and they look pretty good so next thing we have done this as well okay so now let's go towards our right panel and let's design this area as well so I will just change the background color or let's remove this thing as okay so in the right panel first of first of all we'll just create header and then the template area so over here I will just write d. header okay and then we have the Tet container as well so over here I will just write over here header and over here what I will write tablet so I'm just checking spelling so I will just write Adder over here and U the adder background color will be uh what should be the background color we can just WR royal blue and the next thing what we can do is uh we can just set the height as 70 PX and the width will be over here what we can write withth as uh 100% as well and what thing next okay so now you can see that we have cre a header as well that's a bit increase the height of the header we can make it to 90 okay so now you can see it looks very good uh what we can do further over here is we can add a box Shadow as well in the Side Bar and in the header so I will just write Ox Das Shadow okay and if I just cck over here and just MD and reference okay and if you want to use this uh we can simply copy this from here and just okay so just add this over here and just right over here and we can sh St color of joice as well 194 and we can just write r so we can have a bit dark color that's so now you can see that we have a box Shadow I can see that shadow over here now let's copy this and in add in the header as well so you can simply add over here so now you can see the Box Shadow over here as well but it's too much let's make this one forward here so now you can see it's a bit light so now it looks fine so next thing what we can do is uh we will go into the header and just write over here edge1 and I will just write object detection so I will just divide my heading into three different containers so I will just write mty Dash container okay so give it and and over here I will just add the logo so to add the logo I will just write over here with source and I will just have in the my resources folder in the template I have this logo.png so I this logo I will be using over here so I will just go into my resources template logo P so now you can see that uh this logo is not very much Center we need to adjust it as well plus we can also want to change the color of this text as well so for this I will just go to fonts. google.com ions from here and over here I will just click on fonts from here and let's use the this font so I will just go over here and let's go check this details I want to use this so I will just copy this code from here okay and I will just go back over here and I will SL and add add for okay so now I just added this so next thing what I can do over here is I will just go low down from here and just copy this from here and over here I will just write Edge one and I will use this and that color of the text will be white as well so things look pretty good as well so now you can see the color of the text has changed to White uh then what I can just right over here is font size as 1.5 RM so the font size is bit decrease as well so we can set here to RM okay so what we can further do is uh we can just write display flex and okay I will just write just py- content space between align Das items Center okay so now you can see that this looks much good what we can further do over here is um we can change the height to we can change the height as to 30 so if I don't specify any units over here this will be considered as pixels okay so over here we can just write KN D over here okay so now you can see that it has decreased but too much on the right side so we can we can write over here is is in the header as well ping Dash uh right the we can have a 40 BX padding Dash left will be 40 VX okay so now you can see that U now it's very much good and the things looks very much fine now okay so here is my company logo here is the title of the application and this is emps video okay so we are going in ahead in the good direction okay so we have uh cre initial layout so now what I will do over here is in the tablet container I will add a tablet image so uh I will just write over here sources template and this is my where is my template image okay here it is so now you can see it's uh very much uh different so what I will do over here is I will just Define the width and the height so I will just Define width as 680 and I Dash so we can keep the withd as fixed uh we can keep the width as 680 and we can keep the height as 520 okay so this looks much better now so that's fine but we need to center it a bit more so for this what I can just write over here tablet container so what I will just write over here is uh what I will just write display Lex and over here I will just write black St Direction column justify Das content Center and I will just write over here align Das it down Center uh I think there is show okay let me just see over here so we have this okay so I have not defined the width and the height over here so I will just write width as so my width will be the 100% And the height will be 100% of the viewport height so things let me check one thing okay so the spellings were not correct that was I was concerned about so let's see okay so now you can see it's very much in the center but uh we need to remove this it should like we don't want it scroll down and like this so for this I will just right over here display Flex so okay so let's this yeah so what will be the isue over here is uh Sol this issue what I will do here is I just slite over here flection column Aline Das item Center justify Das content Center okay and let's see still there is an so I will just write Le Das Direction column okay so now you can see it looks much good now now it's very much fine and this is what I was expecting actually so now we have the this is basically our tablet area so in open CV if you want to display anything like image if you want to display an image video or like webcam feed we always display played inside a canvas so always remember that in open CV if you want to display anything like uh image video or live webcam feed uh you will display inside a canvas so I will just go towards it now just adding the some sync commments over here okay so now our I will write over here canvas we are just creating a canvas over here and I will just write canvas and inside this I will just add image tag okay so I will just go to resources template and I will just add image. jpg okay and I will just add the ID over here as main Das canvas and over here I will add the ID image we make okay and over here I will add the ID taer image okay so things look pretty good so we have the tablet container tablet ID so let's look it so okay so that's not a good sign so for this we can just write over here well because uh in this uh tablet image inside this this is basically my canvas inside this tablet image and what I will be displaying I will be displaying but whatever I will be displaying either image video or live feed I will be displaying inside this canvas so what I can do over here is I can just write over here position absolute and here I will just write over here tablet image position absolute so this will solve the error so now it looks pretty good so next thing what we want to do is uh we have designed our initial layout so we want to read an image using open CB okay and you want to perform different personalities like converting an image to gray scale and to do object detection we will be using open CV in all that cases so I will just write over here open CV JS and just go to this link over here okay and just click on introduction to open cvj using open CV JS and over here you will just need to click over here so like you can see that here we have all the open cv. JS code so it's currently using version 3.4.0 so if you want to use the latest version you can just write 4. x. but I will be using open cv. JS 4.6 version so I will just write over here this one and here you can see we have all the code so if you want to just save this code you can just typ save as and you can save this file into your desired folder okay so I have already saved this file over here if you just see in my resources I have this open cp. gsy you can ignore this .js file we are not using this in this tutorial so your only concern will be the open cv. GS file which we have I have saved over here okay so here I will just write command add open CV so I will just write script over here okay on CJs and open CP JJs file is basically a 6.8 MB so when we load the web app we don't want the web app application to get stuck or it should wait for the open CV JJs file to get loaded either we want our application to run smoothly it shouldn't wait for the open cv. JS file to get loaded so we can just write as synchronus over here and when the open cv. JS file gets loaded so it will call the open CV ready punch okay so things look all in order so over here I will just write function open CV ready okay and here I will just write console. log open CV ready okay so let's see if it works or not so I will just go to the inspect from here so now you can see we have the open CV ready over here you can grow this fabicon icon so like it works fine okay so we can simply ignore this error open we can see that open CV daddy over here so next what we can do over here is we will just write let image main is equal to cv. imv and over here I just want to read this image and display so I will just write over here uh but let me see what is the image. jpg first I don't want my image Bic we can use this image2.png so we can just write over here image2 do PNG okay so I will just write over here image main okay and over here I will just WR cv. I am show okay and I want to display this image inside the canvas so in open CV I already told you if you want to display an image video live be displayed inside the canvas okay so just write this and just write over here image main dot what is this error let's see like imagy Ma I have made a major mistake so what we can do here is we can just write on run time initialize okay and over here I will just write yeah so it looks fine now so let's see okay but uh that is too much so so now you can see it's covering the complete page because over here I have not defined the width and the height so I will just write over here where of this will be uh so whatever we write over here this will be in the pixels so if I have not specified units over here so I'm not not sping any units over here so this will be considered in pixels so let's see okay but they are not in the correct position so okay so I have found out the issue so this is not correct then this should be over there okay and we can just further this narrow it down so hopefully this issue will be fixed now so now you can see it's look much better so now what issue you can see that the edges are not smooth over here okay so the edges are not smooth so to make them smooth we can simply write over here z- index minus one so over here I just press fres okay right here you cannot see the image so what is the reason so okay okay just give me so the potential is be this okay so if I just change so now you can see over here uh we can see this image and the edges are very much smooth over here so the only thing I did is I just remov the background color from here so it looks perfect now okay so now you can see over here we have the image displayed and let's add different functionalities over here so first of all uh I will just add this over here ID is equal to R GP image okay and over here I will add the ID as gra scale image okay and over here I will add the ID object detection image and over here I will add the ID object detection VD over here so I assigned all the IDS over here so let's first work with the ID rgp image so I will just write over here I do document. get element by ID and over here I will just write over here RGB image okay and I will just write over here on click is equal to function conso log okay and here I will just write rgp image over here okay and the next thing what I will do over here is I will just write let image main is equal to D I am read image M okay and here I will just WR cv. IM show main Das canvas and over here I just image main image do delete so things look pretty good so I just inspect and over here here if I just press this button okay so now you can see over here RP image the number of time I press you can increase see the increase count over here so just copy this code from here I just make minor change and make convert into G scale image B scale image okay and just copy this ID from here as well and just add this add this okay so along with this I will just write that image gray is equal to new c.net so I'm just creating a new instance using CV2 dotm so C using C do C.M we create a matrix or possibly an image so that we can hold a pixel values and to convert the image to gray scale I will just write cv. CBT color here we have the input image and our output image will be saved in image gray and here we have C do Color dash rgpa do ray okay and over here we have this and over here I will just say great so that's loing so this is our RGB image and here you can see we have the gray scale image so we have good to go so next what we will do is we will do the object detection on the image so just copy this code over here but to do object detection we need to add two things over here we've been load that tensorflow JS which is basically required to use the coko SSD model and we will also load the Coco SS model as well so here you can see that I loading tensorflow JS first uh which is required to use the Coco SST model and here I just loading the Coco s s model so be here we are using Coco SS model to do the object detection on images and on the live webcam feed so if you want to get more details over here you can simply write over here penil ljs and you can just uh go to the model section over here so here you can see different models and in this tutorial I'm using uh C model to do multiple object detection on images and on the live webcam feed okay so to load the T to use a model we require tensor flow JS and to do object detection or we require to load the C SST model okay so low tensor flow JS which is that which is required to use the Coco SS model and to do object detction we required the Coco SS model as well okay so just copy this uh come over here and I will just go over here and just write over here object deduction image okay and over here you will just write document or get element by ID okay and over here you will just write on click is equal to function okay and over here you will just write console. log object detection images this okay and over here I will just write for stand image is equal to uh document do get element yd okay so here I will just write uh this part image main over here image main over here so let me write all the complete code of the object detection on image and then I will look back and I will just explain you the complete code I was able to do object detection on image but let me explain the complete code so over here I started writing a code for object detection on image so first of all over here you can see document got get element by ID object detection image so over here you can see here we have the ID object detection on image so what this over here we are doing is that uh this line of code tries to assess an HTML element with ID object detection image okay and here we have on click so when the user clicks on the icon then this the functionality inside this function is implemented so uh when this uh icon is Click object detction image uh in the console we can see that object directional image wrden so if I just refresh this from here so if I just click on this icon so this is the RGB image this is the gr scale image icon and this is the object detction image icon so you can see that object detction of image is being displayed in the console okay so document. get element by ID so this llies uh this line of code tries to assess an HTML element with ID image main so this HTML element over here is this and over here you can see the ID image B okay so here we are just reading the input image uh which we have defined over here so you can see that we are trying to read this input image which we have finded over here okay so now using Coco SST models so I have already loaded the Cs model over here you can see and here I've defined I used the T that here I have loaded the tensor FL JS which is required to use the C SS model so here using COC SS model I'm just loading the C SSD model and then using C SSD model I'm doing detection on the image and what I'm doing over here is that uh in the console.log I am just pred printing all the predictions over here you can see that uh here we are just printing all the prediction so you can see that we have in the image we have predicted around seven different objects which include uh six persons are detected 1 2 3 4 5 six and on one time we have directly the bus so here you can see that we have the class name we have the cence code and over here you can see that we have the bounding box coordinates uh this represent the XY coordinates which represents the top left corner coordinates and here we have this width and the height okay so this is our width and so this is the XY coordinate this is our width and this is the height okay so in this way we have the coordinates for each of the object which we have directed over here you can see we have the class name and the confidence score so using this uh so now you can see that in the prediction do L we have seven different object you can see the length over here as well so now using these uh prediction values what I'm doing that I just created a function draw bonding boxes in so now what I do is uh we go to each each of the predictions like there are seven different prediction we'll go through each of the prediction we will find out the bonding box coordinates over here we will calculate the width and the height we will find out the width and the height we will find out the class name confidence score then I have just created a function compute color of for label so if the class name so if the person is detected then the bonding box will be of this color if the cup is detected then the bonding box will be of this color if the site cell phone is directed then the bonding box will be of this color if not none of these object is detected then the bonding box will be of this color okay so if uh some other object like bus is detected then the bonding box will be of this color if car is detected then the bonding box is of this color so if any other object is DED which is not from this three which is not among these three then that object bonding box will have this color okay then I just want to above the Bing box I want to display the class name and the confidence score here I'm just writing this in the proper format plus my I want my text in the form of cv. f haset deflex so there are other font available as well like Simplex duplex so you can use those as well the font size will be 0.70 if you want to increase the font size you can simply make it 2.70 or 3.70 and if you want to increase the Box uh F thickness as well you can vary this parameter as well so here I'm just calculating the text width so I can just create a rectangle according to the text width with and in that rectangle I'm just adding up my text okay so this way we are doing object detection on the image like you can see over here so this is the gr scale image this is the this is the RGB image this is the gr scale image and here you can see that we are doing object detction on the image so you can see a bit of pause this is that basically the time taking taken by the COC SSD model to load so now we have seen that how we can do object detection on the image in the next part we will see how we can do object object detection on videos as on live webcam feed as well so in the next part we will see how we can do object detection on live webcam feed now we will see how we can do object detection on the life map field first I will just write down all the code then I will explain you all that code as well let's get started I have written all the quote like how we can do object detection on the live webcam feed so over here let me explain you the complete code so here you can see that uh we are just printing in the console when the user clicks on the icon uh in the console this will appear object detection on the like webcam feed and then we have constant video so this uh line of code what this line of code uh does this uh this speciically tries to uh get an HTML element with idid so with of webcam so over here here if you just see over here uh this line uh here you can see that here we have the HTML element video and here we have the ID webcam uh so this line of code basically tries to assess an HTML element with ID webcam then over here in the index.html over here you can see that I've just created a button enable webcam button so that when the user clicks on this button this will enable the live webcam feed so over here in the I have just created a button uh I have in the index.html I have just created a button and ID of this button is enable webcam button so in the main JS using document. getet element by ID so this line of code tries to assess an HTML element with ID enable webcam button and basically it is saved in the variable enable webcam button so over here I'm doing the variable declar declaration over here so we have defined the variable model as undefined we have set the streaming as for FS we have set the source uh we have initialized the variable Source Gap and we have set the FPS value as24 we will be using FPS value later on so in the first step uh we will uh see whether our browser support the when Navigator do media devices object and Navigator dog user media method or not so we are just check doing the browser feature detection we are checking if the our browsers support the media devices uh object and get user uh media method or not so in the next step when the user clicks on the enable webcam button so if the user clicks on the enable webcom button it will first check if the streaming is false so we have initialized streaming is equal to false so as the streaming is equal to false so it will enable the webcam enable the it will go to the enable cam function which we have find over here and it will set the streaming tool and if the user clicks on the enable webcam button again so the this will pause the streaming and the streaming will be uh set to false and the source object of the video element will be set to null as well okay and if our browser does not support the media devices or get uh user media matter so it will generate a warning in the console get user media is not supported in your browser so now uh if the if the user clicks on the button for the first time then it will uh as the streaming is equal to set to fall so this will make the streaming through and the enable cam but function will be called so here we have the enable cam function so if the model is undefined so first it will check the condition uh if the model is undefined so you can see over here we have undef set the model is equal to undefined so if the model it is undefined then it will not enable the webcam stream uh until the model does not gets loaded or defined okay so if the model is undefined then it will not enable the webcam stream stream and uh until the model uh gets loaded okay so it will wait for the model to get loaded then we have the step time out function so over here we have the JavaScript code of the stem set timeout function but what does this code do is basically this try to load the C SSD model uh in in a delay of 0 millisecond so you can see over here we have zero so this this function the step time set time out function try to load the Coco SS mode model with a delay of 0 millisecond so here you can see that set time of punction basically try to load the KU SSD model with a delay of 0 MC and over here when the data gets loaded then we will call the predict webcam function so here you can see that video. Source object is equals to stream so we are assigning the webcam stream to the uh Source object of the video element so we are basically assigning the webcam stream to the source object of the video element and when the data gets loaded then we will call the predict webcam function so here we have the predict webcam function initially in the first step it will check that the our live webcam feed width and the height is greater than zero and in else uh it will generate an error that video dimensions are not worried yet wait for the next frame so in the first step we are just calculating the current time stamp in M second so using constant begin date. now we are just cating the constant uh we are just cating a current time stamp in milliseconds and we are just saving it that time stamp in the variable begin then we have uh we just initialize the variable over here that Source or we have done the variable declaration over here so using new CVM we are just creating a matrix or possibly an image that hold the pixel value and the metrix size is basically uh based on the our width and the height of the live webcam feed and here you can see that uh we have the side the the type of the Matrix will be 8 bit unsigned integer metrix with four number of channels red green blue and the alpha value which is for transparency then using cv. video capture we are capturing the live webcam feed and over here we are using we have just loaded the BSD model over here and the model is saved in the wable model and using model. detect video we are doing uh detections on the live webcam feed and over here what we are doing is that uh we have a draw bonding box function over here so in the draw bonding box function over here we are just looking through each of the prediction one by one and we are drawing bonding boxes and we are just uh adding them confidence for as well as the class name above the Bing box as well so I've already explain this function in the previous part when I was explaining the object protection on image so we are just choosing the same function over here then after each of the frame the to threee of the memory after each of the processing after each of processing each of the frame we are using dot delete function to free up the memory okay so sorry I was just on the wrong side so after processing each of the frame I'm using ss. delete function to free up the memory and over here you can see that we are just calculating the delay so uh over here this is our desired FS okay our desired time and this is our actual time taken so here this is our desired time and here we are just calculating the actual time taken so this is our actual uh time taken to process the gr frame so if the so if the uh desired FPS is uh less than the actual FPS then we will add some DeLay So that it can our actual FPS actual time can reach with the desired time but our actual time is more than the if our actual time is more than or equal to the desired time over here then we don't need to add an delay and it will just go on so here we are just calling the set timeout function so that uh if you don't call this function it will not go from the uh onward from the first frame so to make it go onward from the first frame we are just calling the set diot function so that it will process the further frames and it will do the object detection as well okay so if none of the uh objects are detected then we'll have the S the simple feed coming from the live webcam and if the objects are detected then we will have the bounding boxes around each of the object along with the class name and the confidence score so this is all the go and let me show you a demo so so over here you can see this is the rgp image this is the gr scale image and here we are doing object detection on the image okay and over here now we are doing object detection on the live webcam feed so the model is loaded okay so when the model gets loaded like you can see the message model loaded so when the model gets loaded then we will call the enable webcam function so this know you can see that the live webcam feed is on over here and you can see that it's able to detect Ma phas so over here you can see that over here you can see over here uh now it's able to detect the cell phone over here as well so it is able to detect the cell phone and here I have also have the cup over here let's see okay so now you can see that it's able to DET the cup as well [Music] so sometime it detects that this is a cup and sometime it misses so now you can see over here it is able to detect that this is a cup and the reduction is results are good like our model is able to detect that uh this is a cup and over here now you have can see that uh it's a class and it's cheing by a cup so the reduction results are quite good okay uh the reduction results seem to fine uh over here like you can see over here we have a qu score over here as well and everything seems to work fine okay so if I just click over here it will pause that now the live webcam stream is pa okay and if I just click over again uh the live webcam fre is on again and and if I just click on the pause the webcam Fe is paused and now if I just go over here it has converted image to gr scale and over here uh it is able to do object detection on image and if I just click over here and the model gets loaded after the model gets loaded then you can just click over here and enable the uh live webcam feed over here okay and if you just click on this button the webcam feed is paused and if you click on again the webcam feed is enabled and if you just click over here the webcam feed it is po as well so that's work very good uh next thing one thing we can add is that uh we can add the upload image option over here so the user can upload an image and do the object detection on that image as well okay so I will just go back to the code from here and over here we can just write over here input uh type is equal to file ID is equal to image Dash upload and except is equal to image D SC okay and over here what I will write over here is label for image Dash upload and over here we can simply write I do material Dash icons over here and over here I will just copy this code from here we can just use this same code over here and over here we have the icon name is equal to Cloud upload you can just find the icons name into the font. google.com Das icons I already showed you okay so what we can see see what here is okay so this I cannot see the upload so first of all I will just go over here okay and what here I will just check okay so here we have Cloud upload so that seem to be fine okay so we have the cloud upload option label image upload okay so we can simply write over here as well you can write over here style display n okay so let's see how it works okay so now you can see we have the cloud upload option but it's not visible one change we can make over here is okay so we have I we can have loss equal Circle over here okay nothing changed so so I just need to go copy this from here and just add it over here so I hope this works fine now so let's see so now you can see that we have the upload icon over here so we just will just go to the main.js and just write down the fun over here so I will just add the code over here so I will just add this handle input image F over here and uh in image input function okay and I will just call this function over below over here okay so I will just handle image input and here I will just write document do get element by ID and over here I will just write image upload so just copy this from here and I will just write image upload over here and I will just write on Clay okay oh no I will just add add event list SL okay and here I will just write change okay and over here I will just write hand image input and add okay so now over here let's test this so the user can upload an image so let me upload this image and now you can see that we are able to upload an image so this is the gray scale image and let's see if we are able to do object detection on this image over here now you can see that we are able to do object detection on the image as well we are able to detect the pass persons as well and if I just click over here and when the model gets loaded we are able to do object detection on the live webcam feed as well the model is loaded so now let's click over here so now you can see that we are able to do object detection on the live webcam feed if I just click for here the streaming is paused and we can do again the object detection on the image as well and we can convert the image to gr scale and do object detection on the image as well so that's all from this tutorial thank you for watching this tutorial see you all in the next video till then bye-bye

Channel: Muhammad Moin

Views: 1,282

Keywords: Computer Vision, Object Detection, Computer Vision Web Development, TensorFlow.js, Object Detection in Browser, Learn Object Detection, Computer Vision Web Development Course, Object Detection in the Browser using TensorFlow.js, COCO-SSD, build your own object detection app, deep learning

Id: YSG_541LFng

Length: 80min 49sec (4849 seconds)

Published: Wed Nov 15 2023