How to Build an AI Nutrition Monitor with Python & Google Gemini | Full Code-Along

hello everyone and welcome to another episode of generative geek today we are going to build this nutrition monitor the way this nutrition monitor works is we have a section where you can put in your files so it's like an image of the food item you want to analyze I have a very nice lemon rice here which I want it to analyze it will Analyze This by sending a request to Google Gemini Pro right so behind the scenes it is sending out this image to Google G I Pro and using the prompt that I have written it first gives me the meal name and then gives me a very clear breakdown of everything that it sees and how many calories that item has right and then it says hey this thing has total calories of 767 and whether it's a healthy option or not how much protein it has and what are some important details right it'll be a fun project we going to make use of Google Gemini Pro which is free and we are also going to make use of streamlit in this case so the app that you see here is a streamlit app and U you know it will be a fun project so let's get started so I am in my visual studio code environment and the first thing that we are going to do is create an environment and load all the necessary libraries that are required so we are going to install all the necessary libraries in order to do that uh once I'm here I'm going to create um a fold a file called requirements.txt and um one second I don't know uh why the right click is not working okay uh requirements.txt and here I'm going to say streamlet streamlet is basically the UI of everything that you you see here uh maybe the process has died and therefore we got that error um where including the sections for navigation the way the image is shown way the output is shown all of this is streamlit uh so we'll need streamlit I'm also going to need python. EnV because this is the library that will help us find all the environment variables that we save including the secret keys right that we are going to put in the EnV file um next I'm going to make use of Google generative AI this is the library that has all the necessary functions that make call to Google Gemini so we're going to install this as well once you have done this just do a command shift p you should see python create environment if you don't see it just go to create environment as a search item and you will see it as part of the results right because I use this so often it shows up as the top for me uh and then I'm going to select VN as a virtual environment you select The Interpreter path and then because we already have a requirements.txt I can select this and say okay right so now what is happening is it is creating an environment for me and it is also installing all the necessary libraries that we have asked it to install so streamlet EnV Google Genera AI all of these are now getting installed it generally takes like a minute or two and once it's done uh it'll give you a message saying hey the following environment mment is selected right so the way to check it is go to the terminal and when you go to the terminal I'm just going to bring this up if you see this EnV It generally is an indication that everything is settled right so but I'll still do pip freeze and when you do pip freeze it'll it'll show you what are the various libraries that are installed as part of this environment and all you have to do is find if your libraries are there so I can see Google AI generative I can also see streamlit uh uh can I see streamlit yes I can see streamlit and I can see python. EnV uh python. EnV yes so if I see all of these which were the libraries that I wanted that means my installation has gone all right right so now is the the next step is basically we have our requirements.txt settled I'll just now go out and create app.py right so the app.py is the main file where we are going to do all the coding and um the first step is we're going to install uh import the libraries right so import streamlit as St uh from EnV import load. EnV and find. EnV I'm going to import OS I'm also going to import a few other libraries like you know uh we I haven't imported generative AI so import google. generative AI as gen uh this will be used to make all the calls like I said now because we are dealing with image so I'm going to from pill I'm going to import image uh next what I'm going to do is from EnV I'm just going to load the EnV file but because we don't have the EnV file it will be a good time to now create a EnV file right so create a EnV file and create a variable called Google unor aior key and here within these two codes you should paste in your key I'm going to pause the video here come back once I have pasted my key you should use the Google Gemini API key if you don't know how to get the key there is a video link popping up right now you should use that to basically um go through it to find how you can get a Google API key for yourself right this key is required because without it you will not be able to make calls to Google Gemini so watch that video in case if you don't already know how to get the key so I have pasted the key and now I'm back in that app.py this is now going to be the main file where we're going to do all the coding uh we our intent is very simple we want to rebuild this page completely with all of the items as it's listed here uh so I'm going to now get started with the code right so the first thing that you do when you have this kind of a setup is you say hey you know what we have loaded all the keys now let me quickly give uh this page a configuration right so uh we'll we'll give this page a configuration uh this is part of the stream lit uh so we'll say page config configuration and the way you do it is st. setor pageor config and this is where you will say page title is generative Geeks nutrition monitor right so uh nutrition nutrition monitor right so something like that right uh so this this basically um okay maybe I need to escape this um right so page uncore title is this yeah so page title is generative Geeks nutrition Monitor and then we can even give it some sort of an icon right so let me just bring the icon so we can say hey uh the page icon is um this uh so this becomes my page config my page config is now ready this usually has to be the first line when you are starting with teet uh this has to be the first line um I've have already loaded the environment variables by calling the load. EnV uh next what we'll do is we'll configure generative AI right so gen. configure and this is where you give your uh API key now we I just accepted whatever GitHub copilot gave me but my variable name in the EnV file is not apore key is Google API uncore key so uh whatever variable name you have in that file that's the name of the environment variable that you should basically type here right so API key is os. get EnV this is going to fetch the environment variable from the EnV file that we have right so that's what this is going to do next what we are going to do is we are going to do some sort of uh we'll do a method uh which will basically allow us uh to have um Gemini resp response right so I'm going to say def get Gemini response I'm calling this the method this method will make a call to Google Gemini 1.5 Pro um and it will uh then uh give us the response right so I'm going to say and it's it takes two things it takes one input and the other is the image and I'm just going to say hey model is gen. getet gen ai. generative AI gener ative model and we are going to give the name of the model so the model that I'm going to use is generate Gemini hyphen 1.5 hyphen Pro hyphen latest right so again if you want to understand how to list all the models that are available I'll suggest watch that previous video uh there is a step where you know at this point you can list out all the models that are available and which have generative in their uh name or something right so uh in their meta information so so that way you know Which models allow for generation of text and gini 1.5 Pro latest will even the Flash and all of the those will start showing up there now I've given we have selected the model that we want to use uh now is the time to actually make a call so you'll say model. generate underscore content and this is the point at which you just pass the input and you also pass the image right so uh we are passing this as a list so we are passing the image which is the first element of the image list that we got we are passing the input we are combining them into one list and we are sending that to the model right uh and whatever is the input and you might have guessed the input is largely going to be uh some sort of a prompt right so uh next what we are going to do is we whatever is the response we want to return response. text right so so this method is going to make a call to generate a Gemini and using whatever response it gets it is then going to return that response back to whichever method called it or wherever it was called from right next what I'm going to do is in our thing we also have some sort of an input image setup right that needs to be done so I'm now going to write code for that uh so what we'll do is we'll say def input uncore image unor setup and whatever is the uploaded file I want to process that uploaded file right so um this button will give you whenever you select some file you will get some input file and that file has to be then shown right so so user is going to select the file that file has to be um opened and then displayed here that's the code I'm writing now right so input image setup we're going to say if uploaded underscore file is not none we want to check if there is actually a file if there is a file we want to say hey bytes uncore data is uploaded uncore file. getet value right so so this way you get you get the bytes from it and then you get the parts of the image you will say hey uh these are the parts of the image it's a list so it's mimore type so you will say mimore type and you will pass this as a dictionary where the key value is mimore type and whatever is the uploaded underscore file you get the type from it that becomes the mime type for it and the data is nothing but bytes uncore data right so the bytes underscore data is uh that we have already you know we we upload the file whatever is the file we use we bring get value on it we run get get value on it and we get the uploaded uh bytes right so uh this way you have the the the input image set up what we are now going to do is once this is done we want to return image parts right so we want to return image underscore Parts this is what we want to return back if let's say the nothing was found then we want to raise an error right so the error we want to raise is file not found error no image uploaded right so uh let's just say if this method gets called without any uploaded file or some nonsensical or or a value that doesn't exist then we will this is what we'll get we'll raise right so this is what we are going to raise uh with this this method is done um we have the if else block and the input image setup is done now what I'm going to do is I want to build the sidebar right so so the way I want to do it is I want to build this sidebar that you see the navigation the upload section uh the image name that it's uh the file name that it displays the size and all of that that I want to dis uh I want to build now so I'm going to say hey this is the sidebar navigation that I'm building now and we'll say st. sidebar uh you can give it a title like we have the title called navigation you can give it whatever you want I'm keeping it navigation for now and then we going to say s do sidebar do header and we have a header if you see here it says upload section right so that's the header so I'm going to say upload section um upload section so that's the header and then we are going to say uploaded unor file is st. sidebar file uploader choose an image okay and then it it can even say hey what are the various types that you want to allow if you want to allow jpg JPEG and PNG now now with this basically what is going to happen is this is what is coming in right so you you will get this sort of an uploader uh which will allow you to then um uh upload files right so choose an image and uh uploader my my my thing is going to say choose a image this maybe had some other uh name right so this probably had uh um this probably had upload an image or something like that now let's quickly come here and build this header right so generative Geeks new nutrition monitor our navigation for the sidebar is done I now want to go out and say hey st. header and this is where I'm going to say generative Geeks nutrition monitor uh next what I'm going to do is if uploaded file is not none if uploaded file is not none I'm not accepting whatever it is giving me I'm going to say image is equal to image. open and we are going to uh pass in the uploaded file that way I have the image and now I want to display this image right so what I'm doing is I'm saying hey the file has already been uploaded here I now I have the header ready I now want to have an image area where I can show the image right so I'm reading the image first I have it here now I'm going to say st. image and we are going to pass it the image and we are going to say caption is whatever is the caption you want for in our case if you look at it the caption is uploaded image right so I can also say the same I can say uploaded image uh that becomes my caption and we can we can also say uh use column width as true like you know I'll accept the selection the suggestion it is giving me uh use column width as true next Once you have this uh we will next go out we already we have displayed the image now we want to have like a submit button right so the button that you see here analyze this food that's the button uh sorry that's the that's the button I want this button so let's just go out and say submit is equal to St do button and we can say submit or we can say analyze this food right so uh analyze this food so this will become my submit now is a good time we have the image available we now want some sort of a prompt to come in and that prompt should then kick in with this image and call Gemini right so the method that we have already built now I want to use the the the submit button um whenever this button is clicked I want to call Google Gemini right so so the way to do it is first let's define uh uh first let's define the input prompt right so I'm going to say input uncore prompt and I'm just going to bring the prompt that I have uh so that I don't have to type it in again uh I'm just bringing in the pom prompt and anyways this file is available for you as part of the description so make use of it uh this is the prompt that I have you are an expert nutritionist analyzing the food items in the image start by determining if the image contains food items if the image does not contain any food items clearly State no food items detected in the image and do not provide any calorie information right maybe what I'll do is I'll just quickly uh make it more readable uh by bringing these things in new lines right so every sentence a new line uh that way at least I don't have to uh keep scrolling um yeah any clearly State no food items and do not provide if food items start by naming the meal start by naming the meal identify you can find and then estimate summarize yeah so I have this prompt you are an expert nutritionist analyzing the food items in the image start by determining if the image contains food items if the image does not contain any food items clearly State no food items detected in the image right so there can be a situation where you have this app and somebody just uploads an image of a chair right so uh so you want the llm to to clearly know that if no food item is detected don't go out and start finding calories that are there in Wood Wood right so nobody eats wood um at least nobody in the civilized world that I'm aware of each wood so and do not provide any calorie information if food items are detected start by naming the meal based on the image so we want it to first name the meal right so this is the food analysis meal name that's what I want I want it to First tell me that hey what is the meal name and then go out and list out all the ingredients and their specific calories and then give me a total of the calories right that's what the prompt is in asking it to do so if food items are detected then I want the meal name and name name of the meal ingredient one what is the estimated calorie ingredient two estimated calorie total estimated calories finally mention whether the food is healthy or not and uh food is healthy or not and provide the percentage split of protein carbs and fats in the food item also mention the total fiber content in the food item and any other important details note always identify ingredients and provide an estimated calorie count even if some details are uncertain right so sometimes what can happen is you upload an image and the AI might just come back and say hey you know what I could not identify the ingredient or I cannot respond because I am an AI and things like that and therefore this last point is just to tell it hey always identify ingredient and provide even if some details are uncertain have you always wanted to return back some number right so so this is my input prompt with the input prompt done now we are going to write code for the input button right so whenever the button is pressed you want to say hey if submit with sp. spinner so I want to have like a spinner which says processing so I'll have that kind of a spinner and uh with that spinner running we're going to say image underscore data is equal to input uncore image setup and this is where we're going to pass in uh the input image setup so this is this will become my image data and now I'm going to say response is get Gemini response input prompt and the image data right so you cannot just pass pass the file you have to have this is how Google Gemini wants you to uh pass the image data and therefore you have these image Parts uh this is the way to pass image information to Google Gemini right so get get Gemini response with this now done you will get a response once the response is available what I want to do is I want to first type st. success and we want to say it done right so that's what I wanted to type uh next what I wanted to do is st. subheader I want to say food analysis right so this is this is the part where it says food analysis now we are basically typing the done the done is done now I want to type food analysis and go on with the whatever is the output next we are just dumping whatever is the output right so so I'm going to say st. subheader food analysis and st. WR response right so with this now what is going to happen is my my code in a way is all done right so we have built that entire thing in less than like what 8 81 lines including all the white spaces and you know very liberal use of uh enters and you know uh new lines and all of that uh my code is done now is a good time to just go out and uh run this Command right run this U on the terminal the way you do it is because this is a streamlit app it will say streamlit run app.py right so streamlit run app.py it's going to open um so this is how we are getting it now right so this is how we are getting it I'm just going to uh so it says navigation upload section choose an image this is what we had coded if you remember this is what uh was uh made available as part of uh uh the thing that we were doing um we have no image uploaded uh nutrition analyze this food uploaded image use column width um yeah upload section somewhere we had uh put in um just uh trying to see on on the sidebar choose an image right so on the sidebar we have this so I'm just going to go out and choose some image um it will give me this I'm just selecting the image that I had already given now if you see this um there is no styling on this uh first let's see if the run is all right I'm going to press analyze this food uh so I press analyze this food it's saying processing that's the spinner we just coded this is the spinner code that we wrote uh saying hey once you press this I should see st. spinner processing that's what is happening and now it says done meal name is lemon rice cooked rice and all of that and say 620 calories so with this now um you know your nutrition monitor is ready right so this this thing is ready now you we can go out and say hey you know what let me just write maybe some uh background color and things like that and and uh let me just maybe add some uh custom Styles so uh so I'll just quickly bring in some custom style here uh from my um from the other piece that I have so that I don't have to write it all again um so I'm going to put this custom style right after Google Gemini gen configuration um and then see if it changes the look and feel again or not right so with this I'm now going to again uh refresh the page and you can see my button is now colored the thing has some sort of a background color and that's what the the style information which you can do with st. markdown you can write down all of the HTML that you want here right so but you all but be sure like you know if you if you don't know streamlet always be sure to say unsa allow unsafe htmls unsafe uncore allow uncore HTML to be true only for the HTML that you have written right otherwise this might not render right it might block streamlord might block this HTML from rendering um this this is what we have done I'm now going to bring in some other file and see uh what is the response it gets uh so maybe I'll bring in uh the corn dogs and see if it if it can find uh the name of the food and what this is all about uh so this all of these images honestly I have just found from Google um by doing Google searches it says meal name is corn dogs there's hot dog there's corn butter ketchup mustard all of that and says say 355 calories uh concerning Health this is considered a treat food not a consistently healthy meal and you know what you can do is you can keep playing with the with the prompt and generate this maybe as a table right so you can say hey instead of this um as a as a numbered list can you generate a markdown table for me right so so it will give you uh you will have to play with the prompt and if you do that I'm sure you will find a lot of uh interesting things I have I do the same thing and I recommend you do the same uh so this was it guys if you learned something new if you enjoyed this tutorial um do give me a thumbs up do subscribe do comment do share uh whatever you uh whatever is your feedback I'll be happy to respond to that uh thank you so much again for all your time and God bless you

Channel: Generative Geek

Views: 231

Keywords:

Id: WK8XP5AkuC4

Length: 25min 15sec (1515 seconds)

Published: Tue Jul 02 2024