Image Classification App | Teachable Machine + TensorFlow Lite

in this ig apps tutorial we'll create an image classification app that can distinguish between several objects in my case a banana orange pen and sticky notes we'll create our machine learning model using google's teachable machine then deploy the tensorflow lite model in our android app let's get started we'll head on over to teachablemachine.with google.com which is a web tool that allows us to create machine learning models that are fast and accurate we can click on get started and for the project type we're going to create a new image project that way we can create an image classification app i'll click on standard image model now you can see that we can upload images for our different classes in my case i'm going to be using bananas and other common objects you're welcome to use whatever you want we have the option to add image samples using a webcam or uploading them i'm going to go ahead with the webcam and click on that option now you can see that there's a live feed of my webcam and i'm going to hold on this button to record images of my banana and i'm going to do so from various different distances as well as angles and once i've gotten a sufficient number of banana samples i'm going to move on to my second class which for me is going to be oranges once again you can use whatever you want i'm going to have a total of four classes and i'm going to speed through the process of getting images for those classes now that i have a sufficient number of image samples for my four classes i can click on the train model button to train the model on these images that we collected for our four classes and this can take a couple of seconds so you can pause the video and wait for your model to finish training before continuing when our model has finished training we can see a preview over here in real time so you can see that i have a sticky note displaying and it knows that this is a sticky note i'm going to switch this over to my banana and you can see with about a hundred percent confidence it believes that this is a banana i'll test it one more time with an orange and you can see in real time it knows that that's an orange and if i pan over to my sticky note you know that's a sticky note if i focus in more on the banana it knows that's a banana so now we know that our model is trained and it's working with pretty great uh accuracy and confidence we're ready to export our model that way we can use it in our android app so we'll go ahead and click on this export model button over here and you can see there's different formats we're interested in tensorflow lite for our android application you can keep it on floating point and just press on the download my model button and this can take a couple of minutes so once again you may want to pause the video and continue it when your model has finished downloading so my model has finished downloading and i have this zip file over here i'm ready to head on over to android studio and use this model that we downloaded in our android app so let's switch over to android studio and to add our tensorflow lite model in our android app we're going to go over to our app folder on the left i'm going to right click on it hover over new and scroll down and select other in other we have this option to add a tensorflow lite model that's what i'm going to select now i have to specify the model's location so i'll click on this little file icon over here and my model is in my downloads folder i'm going to unzip this and inside over here you can see that we have labels.txt and model unquant.tf lite which is our model that we downloaded that's already been trained i'm going to rename this to model.tflight and i'll just drag and drop this over here and click open okay now i'm ready to press finish so it's syncing the project and over here for a model of tf light it shows us some sample code for actually using our models for inferencing so in java which is what i'll be using for my android app this is how we create an instance of our model feed it some input and get an output so we'll be using this code in some time i quickly want to show you the layout that i have for the android app that i demoed earlier in this tutorial i'm not going to be going through the process of actually coding the layout because that'll take too much time instead i'll slowly scroll through what i already have over here and you can take some time to copy this down i also have a link to the github repository in this video description so if you don't want to type this all out you can visit that link and copy and paste the code that i put up there on github essentially app will contain an image view displaying the image that the user took after pressing the take picture button we'll also display a text view with the models classification with the highest confidence so for example banana or orange and we'll display the confidences for all four classes as percentages in this textview over here in androidmanifest.xml we want to specify that we need to use the camera so we'll say user's permission android name android.permission.camera here now my mainactivity.java i already have some code and i provided the link to this code that way you don't have to type it out it's in my video description i provided it as a github gist that way you can save some time instead of typing this all out the main thing that we want to do is get the image from this intent called data and then using this image make do the actual inference with our model so that's the main focus of this tutorial and that's why i've provided the link to this code so over here in our if statement to get a we're going to get a bitmap called image using this intent after the user has launched the camera so we'll say a bitmap and then we'll see data.getextras which gives us a bundle and we want to get specifically data so now we have this bitmap called image that contains the image that the user took from the camera what i'm going to do is i'm going to crop it i'm going to center crop it because we want to feed a square image into our neural network our tensorflow lite model so this will say interdimension equals math.min image dot get width and image dot get height so this will give me the smaller of the two of width and height and now i'll say my bitmap is going to be equal to i'll use this some thumbnail utils function called extract thumbnail provide my image my dimension and my dimension so this will perform a center crop for us and now we can we can set our image view to this image by saying imagery that said image bitmap image the last thing that we're going to do regarding altering this image bitmap is we're going to have to resize it to 224x224 because that's what our model takes in as input if you head on over to model.tflight you can see over here the image size it receives is 224x224 and then three is for rgb channels so i'm going to create a field for my image size that i don't have to retype it everywhere i'll call it two i'll set it equal to 224 and i'll say image equals bitmap.createbitmap image and i'll pass in my image size for the width and height oh this should be create scaled bitmap yep and now we're ready to actually do our classification using our model so i'll create a function called classify image and i'll pass in this image bitmap so above my on activity result i'm going to create a void function called classify image its first and only parameter is going to be our bitmap image that we're passing it after the user takes a picture from the camera and this is the part where we can head on over to our model of tf light file where we're provided with some sample code for using our model for making doing predictions and we can copy and paste this into our classify image function so what this line of code over here does is it creates our model and this should be get application context over here we're creating our inputs for the model this is where we actually make our model process that input and we get the output over here as a tensor buffer so you can see that one thing is in red and that's the byte buffer over here that's because we're going to have to take this bitmap image that we got this data for our model that's the input and try to make it into a byte buffer that we can then pass into our model so in order to do this this is going to require a couple of steps the first thing that i'm going to do is i'm going to create a byte buffer called byte buffer and this will be equal to bytebuffer.allocate do allocate direct and i'm going to need four bytes for my float and then i'm going to need some more bytes for the image so remember image size is 224 and then i'm going to need that there's going to be a third i mean a fourth dimension which is the number of channels or g and b so that's why times three so this is how much space we need to allocate to a byte buffer for each image and now i'll say bytebuffer.order byteorder dot native order and one more thing that we have to do is actually get the pixel values from this bitmap image so i'm going to create an array of ins to store those pixel values and because our image size is image size times image size that's going to be the size of this into values array to actually get the pixel values i am going to say image dot get pixels pass in my int values 0 image dot get width 0 and 0 for the x and y image dot get width image dot get height and now into values this array over here will contain the pixel values from our image the last thing that we need to do is actually iterate through this array of values and um add those pixel values to our byte buffer so i'm going to have a variable called pixel to keep track of which pixel we're on i'm going to say for into i equals 0 is less than image size i plus plus for into j equals 0 j is less than image size j plus plus so this is in order to go through to all 224 by 224 pixels in our image which are now stored in int values i'm going to say integral equals into values of pixel plus plus so this will give us the value of the pixel which if you remember is going to be r g and b all rolled into one this is why i've copied these three lines of code over here which are going to perform bitwise operations in order to extract the r g and b values from our pixel so now our byte buffer contains the r g and b values for all the pixels in our image so now we're done with the steps we're creating the tensorbuffer variable that will contain our image as input to the model and we've also have the code for running the model inference and getting the result called output feature 0. using this output feature 0 we can get a float array of the confidences for each of our classes so the way that i'm going to do that is by saying float confidences equals output feature zero dot get floatery and this will in my case give me an array containing four elements now we want our app to display the classification that has the greatest confidence for example if we have an image of a banana and a model thinks that's a banana with 98 confidence we want to display banana in our app in order to do this we're going to have to iterate to our confidences array and find the position with the highest value so this is just going to require some basic code and logic to find the maximum in our array so what this code does is it goes over the confidences for each of the four classes checks if the confidence is greater than the biggest one we've found so far and then it updates the biggest one we found so far as well as the position of the class with the greatest confidence so now max pause is going to be the position in this confidences array with the greatest confidence and now we're going to create a string array of our classes so the order that i took the pictures in on google's teachable machine were banana first then orange followed by pen and then sticky notes and the idea is that i can use this array to easily set my text view called result so result dot set text and this is going to be the class with the greatest confidence and remember we found the position with the greatest confidence it's called max pause so that's why we're here going to say set text classes of max position and the second thing that i'm going to do is i want to display the confidences for all four classes so that's why i'm going to say string s equals it's going to be empty initially and now what this bit of code over here is going to do is it's going to iterate through every single one of our classes display a class name followed by a semicolon and then it's going to display the confidence for that particular class rounded to one decimal place and then followed by decimal sign then it's going to add a new line character for the following classes to come on the lines below now we can go ahead and say confidence which is our confidence text view that set text and then pass in s and then we're ready to run our app and see the results that we get so now i'm running the app on my device and i'm going to click on the take picture button here i have an orange sitting on my desk i'm going to take a picture of it click ok and you can see that's classified as an orange and our confidence's texture displays banana with 0.1 percent confidence and orange with the most at 99.9 percent i'm going to click take picture again and test this on a banana this time i'll actually go close to the banana and see whether the app is still able to recognize what it is and yes it's able to do so with 99.6 confidence i'll test it really quick on the sticky notes i'll skip the pen and here it's detected i classified that as being a sticky note with one hundred percent confidence so in this ij apps tutorial you'll learn two main things how to use teachable machine to train a machine learning model with high accuracy and speed and then how to use a trained tensorflow lite model in your own android app in the upcoming tutorials we'll code our own machine learning model using python and tensorflow and then deploy our custom machine learning model in our own android app if you found this tutorial useful please make sure to give it a like and share it with friends and if you have any questions or tutorial requests make sure to drop them in the comments see you in the next tutorial

Channel: IJ Apps

Views: 79,269

Keywords: IJ Apps, android, android studio, make android apps, Java, machine learning, tensorflow, tensorflow lite, tensorflow android, tensorflow lite android, image classification, tensorflow image classification, android image classification, image classification android app, image classification android, android tensorflow lite, teachable machine, android teachable machine, tflite, image recognition model, tflite image classification, tflite tutorial, teachable machine app, tflite app

Id: jhGm4KDafKU

Length: 15min 19sec (919 seconds)

Published: Wed Nov 03 2021