Cohere Command R+ API / GPT-4 Turbo Vision API Update - Impressive!

today we're going to take a look at the API from coh here that runs the model command r+ uh this has a very interesting integrated web search they call connectors so I built a little script that's going to show you kind of how this works and we're going to take a quick look at the update open AI did on the GPT 4 Turbo API I think we're going to try to solve like a lead code Problem by just using the image so we'll see how that goes so let's just get started so what really caught my attention on the cohere model was that they clocked in at six place here on the LMC chatbot Ara in a leaderboard so that was very impressive above some of the CL 4 models above above Mr Large so yeah I think we had to give it a shot so what I did and I just went over to the coher platform here I signed up you can do that for free so it's really easy to test out here we are going to use the command r+ model so you can try it out for free now you get this free uh API key so what I wanted to try was actually do this uh what they call connectors so basically I think it's some kind of uh retrieval augmented generation system so if you look here this kind of goes out to the web and search and it kind of brings this in as context I think it's using some kind of retrieval here right so if we look at the the python code here you can see we make this message here what is the chemical formula for glucose right and we have something called connectors that is actually using this query here to search on the web to bring in more relevant context so I thought it was pretty cool that is kind of built in into the model and the API so we are going to test that out so let me just head over to the code and I can see you kind of how I set this up so yeah this was a pretty easy setup easy to get going with so yeah we just import uh go here we import EnV and Os we just load ourv file that contains our API key right we have this client here that takes our API key and I created this function called ask question that's going to take in kind of the argument is our question and is going to keep kind of our chat history here and what I wanted here was to create something I called kind of citations so these are all the URLs this uh connector used to kind of find information from our query right so we're just going to do like a simple input here uh the user can do an input and we kind of uh yeah you can call it something like we strip out the URLs we get back from this citation I've called it and we're going to print those urls and other than that we're just going to respond right with the message we get back from the coher model and we just going to run this in a trup so we can kind of ask these questions over and over again and we get the response uh from the chat history I think this append so we actually can kind of keep the context going right so when we ask something we can just follow up with a new question uh about the same topic I'm going to show you a test of that and yeah that's basically it uh you can see this s the chat history here in this list here uh but yeah I was pretty impressed by this to be honest I think it worked pretty good but um yeah uh enough of me talking let's just go out and test it and see what we can get back here so let's open the terminal let's just run this and ask a question so I one want to askina where is Claud three Opus on the leaderboard let's say on the llm sis chatbot Arena leaderboard and hopefully now this will kind of go out to do like a web search through our connector and come up with a good response but also leav in all the citations that we can go and check right if this is a correct answer so yeah you can kind of see here uh on March 27th anthropic clauter Opus L CL mod surpassed opening GPT 400 the chat leaderboard for the first time yeah yeah that's pretty good we get Yow rankings yeah that's a very good answer right and what I like here is yeah you can see we get all these citations here so if we click on one of these you can see we get set over here to a Reddit post uh we can try another one right we can try let's try uh this one and here we get to actually LMC org so yeah that's very good I think this is pretty cool right let's try a different query so let's say I wanted to do some research and find information about the biggest AI conferences in 2024 so let's see what we can get here I haven't tried this so yeah let's see if we can get anything useful here okay so yeah this was pretty good right so you can see here's a list of some of the biggest a conferences so we had yeah some few AI Summit in Dubai NV GTC that's already been AI Summit in London AI Summit in New York and we get all this uh citations here should we test one out let's just try yeah let's try this one I don't know see if it works top 10 AM conferences you must visit it 2024 yeah pretty good so that was cool so yeah pretty impressed by this list here this was neat and tidy and kind of exactly what I was looking for so perfect okay so the final care I wanted to try is can you give me the most important news in AI the last 3 days in a bulleted list format so I thought it could be interesting for some quick research if this actually can give an answer I haven't tried this again so let's see what we can find here okay so we got something back here so you can see here are some of the most important news open Al voice and tool I guess that was last week uh the White House something Google stopped AI from D images of people after that is an old one open ai's new video to Sora that is also an old one so yeah I wouldn't say this is up to date Amazon invested 2.75 billion into anthropic I think that was for some last week so I wouldn't rely too much on this but maybe if I prompted it better we can get a better results and I like that we can see all the uh the citations here so that's cool but yeah I guess this wasn't perfect but maybe with some prompt tuning this could be working well but overall I think this was pretty cool just from this simple script here uh you can kind of get this very interesting result that we can work more on if we wanted to right so I'm definitely going to explore this model more uh again I'm going to just going to put this uh all these scripts I'm using today just follow the link in the description you can find them on GitHub if you want to try them yourself so just go check that out in the link in the description now let's go take a quick quick look at the gp4 turbo update so opening I put out this tweet yesterday major improved gp4 turbo model available now in the API and rolling out into chbt uh so majorly improv that's very vague right uh but again I went ahead I take a look at this platform here and you can see they kind of update the gp4 turbo model here uh so now it kind of has gp4 Turbo with wish I think that's pretty cool so we can kind of now do everything in one request before we had to do it in like we had this GPT 4 Vision preview so we had to kind of combine the the other GPT API call with the wish preview API call that was a whole mess but now we kind of get it into one model here so I think that's a good update for developers right so I think it's pretty cool so I went ahead I created a script for this so let's check it out so yeah it's pretty straightforward uh yeah you can see what we need here we just load our EnV file we put the open I key in V we have the client uh I don't want to do my images from my URL I want to do it from an image path so we going to encode this into base 64 right and I have this function called ask about image where we take our image path and our question as arguments and yeah we're going to use the new GPT for Turbo model right and we have the user role the question is going to be put in here and we're going to feed kind of our URL uh with our base 64 image here and yeah set my token to Max th000 I have an image uh a function here to just open the image path so I want to show the image on screen when we ask about it I set my image path and I open the image and we have this input where we can ask questions about the image this is not in a loop but yeah we can just start it over again and it's going to run the ask image function take the image path take the question and kind of yeah we print the response for that so let's check it out pretty easy setup but it works yeah as intended okay so the first thing I wanted to test was I went over here to lead code and I just took like a screenshot of the full uh the question here from just a context you can see yeah we have the screenshot of the the question here of the problem and yeah I just set it to lead. PNG here and let's go to terminal and run this now and yeah you can see the image pops up here and then we can start asking questions about it so let's go ahead and just do can you solve the lead code and give me the answer in Python 3 syntax so let's see if we can actually solve this lead code Problem by just putting in image asking this query hopefully we get a code back let's copy it paste it in and see if it can solve the the problem okay so we got an answer here let me zoom out a bit yeah here's the possible python solution so let's cop this right let's go back here and let's paste it in here uh yeah let's zoom out a bit and let's run it so we can see test results down here accepted okay yeah so we got all three cases uh I guess that was an easy problem but still yeah that worked pretty good so I guess it can solve these lead code problems uh just from a screenshot okay so next up I thought I just gave it an it hasn't seen before so I went to paint I created this masterpiece here we have a guy on an island here with some tools and some trees we have some Shark in the Water we have a boat on this other Island right so I screenshoted this image and let's run it now you can see image is probably going to pop up here right and let's ask uh what can John juice on this island to survive okay so this was good in image JN is is simply small island surrounded by water and sharks for survival JN can use these resources there's a boat likely the most practical asset JN is the boat we have some trees source for food wood for building serious sources uh fishing could be available to find food water collect rain water okay shelter using wood from the trees like leaves or stone okay so it doesn't say anything about the tools but uh safety from the Sharks when using the boat could be crucial would be crucial so assessing the best times to travel possibly when sharks are less active would be helpful okay so that was pretty cool that was just a fun example I wanted to try out uh just by drawing this image myself uh so yeah I think again this script works good I kind of like that open AI kind of changed it so we can kind of just make like a everything in like a single call here so we can kind of do the image URL object here and we instead of having two different models we have to call that's just going to increase the latency right and stuff uh but yeah uh as I said you can find both of the scripts we use today in the link description below to my public GitHub uh check it out if you want to uh yeah bit of a different video today I got a project planned for Sunday I think so we will see about that but uh other than that thank you for tuning in and I'll see you again oh yeah Sunday

Channel: All About AI

Views: 5,554

Keywords: cohere tutorial, cohere api, command r+ api, cohere command r, gpt 4 turbo api, command r api, ai engineer, ai engineering, gpt 4 vision, gpt 4 vision api, python

Id: uLvk5jeKccc

Length: 11min 54sec (714 seconds)

Published: Wed Apr 10 2024