9 Mind Blowing Things You Can Do with ChatGPT-4 Vision (ChatGPT-4V)

hello there my name is Gary Sims and this is Gary explains now a little while ago open AI announced a new version of chat GPT chap GPT 4V or 4 Vision a version that can actually understand images so you can upload an image a photo you uplo some text you can upload math problems and then it can analyze that and answer questions on the image that you've uploaded now it's only available to chat GPT Plus account holders and it's been rolling out quite slowly and finally finally it was enabled in my account so I've been giving it a try and playing around with it so here are nine things that you can do a chat GPT for vision so if you want to find out more please let me explain okay so here we are inside of chat GPT I have chat GPT plus and take access to GPT for vision you need to have a Plus subscription and also make sure you don't it's not available with any of these plugins you have to be on the default there and you should get this little ability here to attach images so before you can write text now you can attach images and that's what we're going to be doing today so let's start with something simple so what I've attached here is a picture of a house plant now to me that looks like a cabage growing in a house plant which surprised me a bit which is why I took a photograph of it but let's ask uh chat GPT with a vision what I'm actually looking at so what is this plant question mark so it's going to away there's the image and it's going to think about it so it's an ornamental kale or ornamental cabbage these plants are known for their vibrant and colorful leaves and they are often used for decorative purposes in gardens and Landscapes especially in the cooler months so there you go so it's not just a cabbage it's an ornamental kale or ornamental cabbage so I learned something and I guess that chat GPT got that right now one of the amazing abilities of this Vision version of chat GPT is that it can read handwritten notes handwriting handwritten diagram so here I've got a flowchart and as you can see it's basically a simple Loop it says I to zero is I less than 10 yes print I increment I go back up to the top no then you end that's really badly drawn I mean that is not nicely drawn in any way whatsoever neither is the handwriting that good let's see if chat GPT can turn that into some python code write the python code for this flowchart the flow chart appears to describe a simple Loop where I is initialized to zero correct there's a check if I is less than 10 that's correct then the program prints out the value of I and increments I by one that's all correct this continues until I is no longer less than 10 at which point the program ends absolutely absolutely spot on at the correct description of it and now it's giving us some python code let's see while I is less than 10 print I increment I that's it it's a very simple Loop and so there you go so this is brilliant this has just converted badly handrawn uh flowchart into some python code we've got some more examples of it reading my handwriting uh or our handwriting in general in a minute let's try something else okay so this is a chart I used in my recent video about the tensor G3 this is the geekbench 6 multicore scores what I'm going to ask it to do is convert this into a table make a table from this data here's a table representing the data from the image phone score that's pretty good y so phone and score tensor G1 pixel 6 3,300 tens well that's reading that pretty easily and it's giving us the table so here is a good way of being able to take a a graph that you that has all the data you want on it and then get to convert that into a table for you uh and then you can use that uh for whatever you need it for so that's that's doing the reverse sometimes we talk with a table and say generate me a graph this is actually taking the graph and giving you back the table that's pretty impressive so now I'm going to give it a math puzzle a visual math puzzle this is actually quite uh complicated it certainly does take you a few moments to realize what's going on the question is is each firework a to F contains six stars which firework has not been launched knowing that each one gives us one two or three stars of its corresponding color so we'll cut and paste that into the actual question and see what it can do okay so what it's done is it's gone through and it's analyzed each firework firework a what it tells you the number of stars that each one can do fair enough that's not actually right one blue two red three green that's not actually uh right actually because firework a is one green two red three blue but it's funny enough it goes through it all and then it counts the number of stars that there are and then it works out that the number of blue stars is right the number of green stars is right but it says based B on the star count fireworks C has not been was actually the right answer that's actually the right answer fireworks C hasn't been done but it's not because of the red stars it's because of the yellow stars so where did it get that wrong for C it says there are one yellow two green three red well see actually one green two red three yellow so there you go it's got the colors wrong uh and therefore it's misunderstood but actually because it's worked out that it was it said it had three red stars actually cuz it's got three yellow stars the yellow ones are missing but we could have got the colors all mixed up it still come up with the right answer so there you go that's pretty impressive even though it didn't get there by the right route it got the right answer okay so the next thing uploaded here is a us to Euro currency conversion chart covers a period of 1 year from October 2022 to October 2023 let's ask chat GP to describe what it is what am I looking at question mark so you're looking at a chart that displays the exchange rate between US Dollars and Euros over a one pit exactly what I just said okay it's breaking it down a bit it's giving me the title so it's read that text off the image remember there's no text in that image it's just an image and it's actually it's doing optical character recognition to read that okay and let's see if it comes up with any more interesting information as it digests what's on this image okay so it's gone ahead and described that chart in pretty good accuracy giving me you know every single aspect of what the different you know the volume chart and the graph and you know the the different values that are shown on the chart and the dates and everything so it's read that uh really well I I'll ask it this question I don't know whether it would mean you know what what it means let's see if it understands what does that mean about the value of the US dollar compared to the euro let's just see whether it it comes up with some kind of analysis of this I am not expecting it to to do anything CH indicates that the value of the US dollar has decreased relative to the euro uh over the past year here's what it means depreciation the 7.63% decline mentioned in the chart title suggest the US dollar has depreciated or weakened against the Euro by the percentage over the year okay so there you go so it's now starting to look at this purchasing power what does it mean if I was purchasing in you know dollars compared to to euros and so on so that's pretty interesting so I I'm pretty impressed that it could do that as well obviously let's just throw this in here you should never take Financial advice or medical advice from uh these uh large language modules like chat GPT that's not what you should do always seek out professionals well that's my little thing there to make sure that no one does anything they shouldn't do okay let's move on to the next test now if you ever do any family history research you will come across documents like this is a census document from the UK 1851 I think and for me the family that I'm interested in here is the low family though uh some people would transcribe this as love but that's actually a w not just a single V we'll see what uh chat GPT makes of it but it'd be interesting to get chat GPT to transcribe this for me to put this into you know a table or something so let's ask it to do that transcribe the text in this image the image seems to be a historic record or sensors that's correct here's a transcription of the visible parts of the image and so now it's going ahead and reading that for me let's see how it does okay so it's created a nice table with all the different columns there yes it's come up with love Thomas love Elizabeth love it's actually low but that's fine even human uh transcribers make that same uh error it's easy to understand why but seems to be the rest of the stuff head of the house wife daughter uh married unmarried that all seems to be working age so that's really interesting and it's coming through and giving all of those people so there you go that's really really interesting and I I've struggled many times and I've seen some of these documents to try to read them to try to work out uh what exactly is that written there because people write them in a rush and they write them in this very cursive handwriting and he's doing a pretty good job there of transcribing that for me just that one mistake putting love rather than low but I'll accept that now here's an AI generated picture funny enough uh as you can see there's two two moons or one and a half moons in there let's ask chat GPT what is it that we're seeing please to describe describe this image the image depicts a dramatic and surreal Seascape at the Forefront tumultuous waves crash with their froy white Crest contrasting against the deep blue of the ocean there are two lighthouses in the scene one stands prominently in the middle tall and striped with alternating red and white bands its Lantern room at the top is lit casting a soft glow another light housee slightly shorter and simpler in design is positioned a bit behind the first one and so on so there you go so it's giving out some information about the picture to add to the Ser another Celestial body resembling a planet or smaller moon is visible to the right side of the image nearer to the Horizon a solitary bird possibly a seagull is captured in flat and so on so it's pretty good at describing uh what it's seeing so top marks there so here's another handwriting one one I'm showing here on the left is an unbalanced binary tree an unbalanced AVL tree and then on the right is having done a right rotation it it creates the balance version now this is quite a complicated subject and there's left rotates and right rotates and so on I want to ask chat GPT if it understands what this picture is and give me maybe a lesson plan for a high school computer science class based on this image so let's see what it can do with that create a lesson plan for a high school computer science class based on this image okay so it's gone ahead and created quite a comprehensive uh lesson plan by the end of this lesson students will be able to understand the concept of balanced and unbalanced binary trees and demonstrate the ability to perform a right rotation to balance an unbalanced tree well that's exactly what that diagram is showing so it says you need some graph paper and some pens and paper start with a discussion on trees in computer science explain what binary trees are and how they are structured in the concept of balance and unbalanced then it goes on through about how you talk about the term unbalanced and uh right Road R ation and then even there's a group activity it's given me for 10 minutes divide into and give them a different unbalanced Tree on paper after each group div finding the right rotation to balance the tree there's a practical explanation of doing these all about the right rotation and it does say at the very end here uh that it it should be good to look at left rotations so it does actually give a pretty good plan here just based on that diagram which was just a drawing I wrote a few words on it and it's come up with a pretty good uh lesson plan even with the number of minutes I would need and the activities for the class absolutely brilliant so well done there chat GPT okay one last test I'll give it this yellow blank image and see what it says just read what you see oh it is blank by the way I love the G Spain Channel well that's very kind of you to say chat GPT I I enjoy using chat GPT as well I'm glad we are friends well of course what have I done there I've actually written it is blank by the way I love the Gary explainers channel in a slightly different color of yellow on that blank image there and probably with the video encoding and so on you can't see it so it looks like it's a blank yellow uh image but in fact it's got that word written on it so there you go uh by the way I love the G explanation well so do I so there you go chat GPT for vision I thought that was pretty amazing especially the fact I can just scribble something down on a piece of paper feed it into chat GPT and out comes a lesson plan or a transcription of what I've written absolutely amazing love to know what you think about it which was your your favorite use case do you have an interesting use case that you have thought about do you have chat GPT plus all these are great things that I'd love to discuss with you in the comments below okay that's it I'll see you in the next one

Channel: Gary Explains

Views: 15,666

Keywords: Gary Explains, Tech, Explanation, Tutorial, ChatGPT, Open AI, OpenAI, ChatGPT-4V, ChatGPT-4 Vision, ChatGPT image capabilities, GPT-4V(ision), GPT-4 with vision, analyze images

Id: qw1Wyiv2F3Q

Length: 13min 32sec (812 seconds)

Published: Tue Oct 17 2023