Object Detection android app | deep learning android | tflite | mobilenet

hey everyone how's going on welcome back to another video and in this video we are going to learn how you can create a object detection Android app so the idea is very simple we are going to use this model and the input will be the image and the output is going to be the bounding box around the object and the class or the label of that particular object detection so we are going to build this app in this video and if you are interested in making something like that so welcome first of all we need to download the startup model with metadata so you just click on this download the starter model and I will leave the link to this page in down description now our work over here is done now let's open our Android studio and start by creating a new app all right so here is our Android Studio let's create a new project an empty activity and let's give the name let's just say object ml app and hit enter the language I am going to use is Godly language if you want with Java just let me know in down comment section well there not be a much difference in it you can just relate different languages but if you want with Java just let me know in down comment section I will make one video for that also alright our Android Studio is now ready now it's just ready to write our code so first of all let's get to our UI part we are going to create very simple UI so UI wise I'm going to use a relative layout like that and we are going to have a image view so let's just create a let's have a image View and it will be match parent height and bits will be match parent let's give ID as image View and let's just give a background also to it just to see everything is going on fine or not and other than that I guess everything is fine and now we are going to have a button so let's just give a wrap content wrap content and let's just give it a text uh let's see let image and it's going to be attached to bottom so layout bottom true and let's just align Center horizontally let's make it true and let's just give a margin bottom of let's say 20 DB and I'll just also change its background tint a little bit let's just say black for now and for image I will just make it this color and for button I will make it with this orange color okay let's just change this one okay this is our UI if you want to change it just go on with that now the main game begins on mainactivity.kt so let's just first uh declare our button and image view real quick so here we have a image View and also the button and I'm going to also use a bitmap variable to store the image you will just see where we are going to use it for now let's just declare it over here now let's initialize our image view image view is equal to find view by ID r dot ID dot image View and similarly button is equals to find view by ID r dot ID dot button I forgot to give the ID to the button so let's just give the ID let's say BTN real quick over here now let's get back to our main activity and just say now the main idea is when the button is pressed we want to open the gallery and just allow the user to select the image from file manager so when the user selects any image we will use our model to detect the location of the object and just draw the rectangles and prediction on that bitmap so let's just do that thing first of all when the button is pressed we are going to say set on click listener so when the button is pressed we are going to use the intent so let's just say where let's just Define the intent over here only let's say well you intent is equals to intent to import hit option enter and Now intent okay so here is our intent now let's set the type of our intent so intent dot set type so the type is going to be image so the type of the data we want to show to the user is the image data now finally intent dot set action so the action is going to be action get content so intent Dot action get content that's it now when the button is pressed we want to uh start this instance we'll say start activity for result and just pass this intent over here and just give it a code let's say 101 now out of this when the user selects any image one callback function is invoked which is on activity result so we just overwrite that function like this now over here we are going to first check if the request code is equal to 101 which we have passed right over here now if the request code is that we are going to use this data to take the image from that whatever location over here inside this data so first of all we need to fetch the location so we'll say where URI which is uniform resource indicator and you can say data dot data now inside this URI it is the location where the image is located so we need to access that image and store that image inside this bitmap so that's why we have uh used the bitmap variable up there so we'll say bitmap is equals to mediastore dot images dot media dot get bitmap and just have to pass the content resolver which is this Dot content resolver and pass this URI just like that now we have the bitmap inside this bitmap variable now once we have the image we want to run the prediction on this image right so we'll just create a function get prediction so we'll just write this function get predictions so fun get predictions okay so all right now is the time to import our model so which we have downloaded so for that you just have to click on file new and just go below click on other and we have to say tensorflow Lite model now you have to click on this folder icon over here and locate the model wherever it is located in our case it's this SSD and soul mobile net model so you just hit open and hit finish that's all now let's just uh import the labels.txt also with this model you are not going to get the labels.txt for that you can just go below and down description so you will just get access to that levels.txt from there so for copying the labels.exe first of all let me just show you all right so here is the labor start txt which which you can get in down in description link will be there so just copy this and uh get inside over here just you can just click on this project and then project over here now go inside app Source Main and create a new directory new um directory and you can name this directory as it's now inside this assets just paste this labels.txt file let's just change it back to Android okay now once our model is loaded over here you can see it over here if it is not opened automatically for you you can just go back to this project then ML and double click on this and you can get access to this code once again so I'll just copy this code and paste in our get prediction function for now and let's say option enter wherever you are getting this red mark for tensor image option enter and instead of this context we'll see this now I'm going to uh declare this particular installation of model up here this will be my model so since I want to access it globally so I will just say delete init where model what is the type of it it is this type instead of closing the model over here I am going to override a function which is on Destroy when the app is closed then I will close the model so I'll just get rid of this comment sections also foreign all right so everything looks fine now it's time to load our labels.txt also so for levels.txt I'll just do the same I'll declare let's just uh get it over here only so we'll see value labels equal to so this time we are going to use file util util don't uh don't use this file utils you are going to use file util hit Dot and if you are getting this import error hit option and enter this file util is from tensorflow given to us uh this over here you can see so we are going to use file util to open our file util dot load labels now you have to pass the context which is this and then the name of the file which is labels Dot txt this is going to return you a list of all the labels with the index particular index so this is how you can read the reverse.txt very easily by using tensorflow all right everything looks fine and now the only thing we need to change is that this model expect the input to be in the shape of 300 by 300 but our bitmap which we are passing over here could be of any size so we need to resize this uh image or this bitmap we have two options so I'm just going to resize this image with the help of image processor which is again provided to us by tensorflow so I'll just say uh that let's just say well image processor equal to image processor so let's just import it option enter and we are just going to say Builder dot add and we are just going to add resize op and if you are getting import error hit option enter and height and width will be 3 by 300 and Method will be resize math third will be bilinear and just plot it build like this now we are going to use this image processor to resize this image so we need to convert this to where with the image equal to image processor dot process pass this image like this all right looks fine over here okay it should run fine up till over here but we are not showing the predictions anywhere right so we need to use this uh outputs in order to show the prediction so first of all these are load buffer so we'll just convert them to float array so that we can access them easily so all of them like this okay now let's have a look at the outputs uh what the output look like which we have over here location classes score and number of detection so let's just get back to over here so first of all number of detection is the number of objects it is able to detect in the image which you have passed this will be a single value this is not going to be a array now second we have the classes so the classes is basically the idea of uh all the objects it has detected inside the image which I have passed so this is going to be a array and scores is the confidence of every object which it has detected and location is self-explanatory but it is the location of the object inside the image we'll use these locations to draw the rectangle we'll use this classes to get the ID to get the name of the object it detected by using the labels which we have imported and this course we are going to use to get the confidence of that particular detection so let's just do the same use all of these outputs in order to draw on our image on our bitmap so for drawing we are going to use canvas and paint so let's just go to the top over here and Define the paint value paint equal to paint uh just like that and other than that we are going to use the N canvas so we'll see value canvas equal to canvas and we need to pass the bitmap to it so I'll just hit import now if I pass this bitmap like this it's going to give me a error since this bitmap is not mutable so we need to create a mutable bitmap so I'll just say mutable over here so for creating a mutable bitmap it's very easy you just have to say where uh mutable is equal to this mutable the name of bitmap so we are just going to say bitmap dot copy and then configuration will be argb 88888 and it will be true is mutable will be true now this mutable I am going to pass to this canvas and on this canvas you can just use the features like draw rectangle draw text or put text and there are a lot of drawings you can do on this canvas now so now let's just iterate inside this course so I'll just say scores dot for each indexed this one I'm going to say canvas dot draw a rectangle but before we draw let me just tell you that this locations contain the location normalized value from 0 to 1 so we need to scale this value to whatever height and width of the image is so we need to get the height and width of the image also so we'll save value height is equal to uh mutable dot height and value gets equal to mutable dot width so we'll just multiply with height and width to scale the image and other than that I guess everything is fine I'm going to use x variable also let's just okay so you will see if x is forward but for now let's just say x is equals to index over here and now we are going to say if f l is greater than 0.5 since we are iterating inside this score so if the score is greater than 50 percent then only we are going to consider that as a good prediction right so we'll say draw rectangle canvas dot draw rectangle so it is going to take the rect F object and paint paint we have it so it's just this paint now it need a direct F object now this react F needs four coordinates uh the top left right and the bottom right one so it needs four coordinates which we can get from these locations from 0 1 2 3 is the location of the first object then four five six seven is the location of the next object so we are just going to say X over here and from this locations we can say locations dot get we are going to get X Plus 1 first it's just how it stores the value so locations dot get X then then we'll say locations dot get X Plus 3 and then locations dot get X plus four uh sorry X plus 2 so 0 1 then 2 and then we have three now I'm going to scale them so so this is my uh x value so it will be multiplied by width and the other value will be multiplied by height this is uh I guess the left one so multiplied by width and this is the top one so multiply by height and similarly multiply by width and multiply by height that's it uh one more thing actually this paint uh this paint is going to draw the rectangle field one we need to draw the rectangle stroke one so we'll just set the style so tile is going to B is equal to paint actually style dot stroke so we need to draw the outline rectangle not the field one now we are going to draw the class whatever the class of that particular detection is and as well as the confidence Associated to it so again I'll just change the style to field one since we we need to write the text we don't want a stroked text we want the field one so we'll just change it to fill so now we say canvas draw text so it need the text XY value and then paint text will be inside the labels and we want to get from the labels we want to get the classes and from the classes we want to get this particular index and this is a float so we'll just convert it to int now it needs the location X Y location and then finally the paint so let's get the XY location XY location is going to be the same this one okay so it is only going to write the label of it we also want the confidence so we'll just get this FL value like this I'll just convert it to string just to be safe yeah I guess everything looks fine and it should run fine except one uh which is this x so this is equal to index so index is going to be 0 for the first case it is going to run perfectly fine but when the index will be 1 x will be 1 so here you see one two three four so we don't want one two three four we have already 0 1 2 3 we have already used 0 1 2 3 initial even index is zero then we want four five six seven right so we will uh get the X like that and we will multiply by 4 so when X will be 1 meaning when index will be 1 it's it is going to start from four five six seven when index is 2 so it will be 4 times 2 which is 8 then 8 9 10 11 right I hope it makes sense so now I will just uh check once again if there is any problem or not and then we are going to run our code okay our app is crashed let me just check what is the problem it says that attempt to invoke virtual method on resources and my guess is that this line might be causing the error so I'll just comment this line for now and for the labels I'll just get rid of this for now let's try to run app once again and see if that is the problem yes that was the problem uh but why so file your deal dot load labels this dot path okay so I'm thinking let's just say late in it where labels over here so I'll just Define these labels down over there but wait a minute so I'll just redefine this thing over here and instead of uh saying this I just select in it and it wears like this I will copy this remove this and it is going to be a list of type string and on create function here I'm going to Define this and hopefully this would solve the problem let's run this once again yes our app is running fine now okay so let's try making some predictions from our app so I'll just click on select image and let's select this for uh okay okay okay I'm sorry guys actually actually one thing I forgot we are drawing on the image but we are not showing it on the screen so we'll just say image view dot set image bit map and it will be a mutable let's run this once again and this time it should solve a problem all right so let's click on select image once again let's select this one okay actually it is detecting something it says Apple um let's try to get this and it says dog but the problem is this font size everything is so small so let's just solve that problem also so for that it's not that difficult actually we are going to access the paint one so paint dot actually first of all let's say text size so the text size is going to be let's just make it proportional to the height if possible so we'll say height divide by let's just say 35 f and similarly this is the rectangle which is the stroke width so spent dot stroke width is going to be height divided by let's say 85 F let's run this once again and let's hit select image and this time again dog the stroke is fine but the text is still too small let's just say 15f if let's again select image and let's select the dog okay dog with 76 accuracy everything is looks looking fine right now let's select this apple it says apple with 71 accuracy fine let's select this bottle is it at two percent accuracy let's try uh this one everything it says that person person it is detecting person fine and let's select this one it is detecting it as book and this has cell phone and this has Mouse now I'll just make very small difference over here okay so what I have done I have created list of colors so for every prediction um here for every drawing we are making I am going to set the color paint so I'll say paint dot set color so the index will be the sorry if not the index actually colors is the name of list I have created you get the index value from it let's run this once again now if I select the image and something with a lot of so you see that everyday tangle has some different color red green uh pink red then again yellow so this is the change I have made and very small little change but looks good over here let's try to select something else so here is a book cell phone and mouse let's try to select this is blue and which is banana is 75 accuracy and here's the cat with 73 accuracy all right guys so I think it is enough for this video and if you have any queries about this video this project this application you can ask them in down comment section and I'll end this video right over here and if you found this video helpful make sure you hit the Subscribe button and share it with your friends and also like this video so goodbye I'll catch you in the next video

Channel: ProgrammingHut

Views: 18,420

Keywords: android app tflite, object detection android app, tflite object detection, detect object in image, with location, android studio, kotlin, java, ml deploy, dl deploy

Id: RDARCjVpg8Q

Length: 23min 50sec (1430 seconds)

Published: Wed Dec 28 2022