GPT 4 Got Upgraded - Code Interpreter (ft. Image Editing, MP4s, 3D Plots, Data Analytics and more!)

I just got access to the code interpreter plugin about 48 hours ago and I've been running experiments on it non-stop since then I've come up with about 18 examples to show you guys its power most of them I reckon haven't been seen before I predict many Industries will have to update overnight when it's released more widely and at the end of the video please let me know what you think and what other experiments that we can try first though what about this one a 3D surface plot just quickly the way it works is you click this little button to the left of the text box and then you can upload many different file types like CSV files Word files images and even short videos then it will automatically analyze the file type without you pressing anything and then of course you give it a prompt and as with all of chatbt it becomes a conversation so the first 3D surface plot was decent but it was too small so I simply said in natural language can you make it four times bigger thank you and of course you have seen the amazing end result even with the lighting look at the shadow is there I believe this is based on a real contour map of a volcano in New Zealand and I could do a whole video just on this but I have 17 other examples to get to but this one was truly amazing did you know for example it can generate QR codes I said create a QR code that I can scan with my phone to reach the following URL and lo and behold it creates it and yes it does work maybe I'm easily impressed but I think that's pretty amazing and what about a 3D scatter plot this is truly remarkable I uploaded the data from gapminder and it created this chart based on the median age of over a hundred countries from 1950 I think projected to 2100 and I asked highlight the UK this is indeed the UK's median age through those years in red but I know what you might be thinking that is amazing that it's 3D and interactive but the blue kind of merges and it's hard to see what's going on I engage in a conversation and look what it created it picked out the 30 most populous countries and separated them off with separate colors look at that that is gorgeous now you might have the critique that the median age is in descending order in the y-axis going from 20 down to 60 so in a sense the median age is actually Rising not falling but nevertheless that's easily amendable and that is truly an incredible diagram and look just for fun I'm going to go into the data look at this I'm traveling into the data this is so wild I don't know how helpful it is but I think that's just beautiful and crazy there are so many Industries data analytics accounting consultancy that this will affect by the way it got all of this done in about a minute I see a lot of people online talking about five seconds later it is no way done in five seconds you have to wait 30 seconds a minute sometimes much longer before I move on I want to give you a killer tip that it took me quite a while to work out so when you get access try to remember this say output the visualization as a downloadable file if you don't add that phrase as a downloadable file what will happen is it often gets stuck at this stage of the code it'll either say fig.show or plot.show and then just stop I found that I encountered this problem far less often if I said output a downloadable file next did you know that code interpreter can do optical character recognition I screenshotted this text from a New York Times article I think it was and I asked OCR the text in this image and write a poem in Danish about it now I don't want to exaggerate it often gets OCR wrong I don't want to get your hopes up it fails more often than it succeeds but when it works it can do it understood the text and then did a poem in Danish about the text now I'm going to need a Danish speaker to tell me if that was a good poem but either way it could do it how about this one it can do interactive time series with range sliders and selectors I uploaded a CSV file on life expectancy data from the entire world and I just said can you pick out the U.S UK and India and create a Time series with range slider and selectors again that killer phrase output a downloadable file and here is what it came up with notice how the life expectancy for all three countries Rises during the 20th century and look how I can select down here interactively a range of the data and even by clicking up here a 10-year interval or 50-year interval but here's the crazy thing I did nothing I just uploaded the file there were hundreds of countries in there you can see here all the steps that it did and if you click on the Arrow you get to see the actual code then it goes through shows its explanation and eventually gives you a link that you can simply click and get the file downloaded and if you weren't that impressed already here's where it gets fairly game changing you can get it to do the data analytics not just the visualizations for example I said find five unexpected non-obvious insights from this data and offer plausible explanations for them this was bad to the median age data for the most interesting observation provide a compelling and clear visualization now ignore the first diagram which wasn't that good because of the x-axis but look at the insights this is data analytics you can see here that the original file was called median age years and it was just a table of data no analysis whatsoever but look what gpt4 picked out in site one the global median age has been steadily increasing over time it calculated the global median age that wasn't included in the data it was just country data and it says it's gone from around 22 years to over 38 years in 2023 and it's projected to continue rising to approximately 44 years by 2100 and then it offers a cogent explanation this trend is likely due to a combination of increasing life expectancy and decreasing fertility rates worldwide as Medical Technology improves more people are living longer birth rates are declining particularly in developed regions is pick this all out and then it moves on to the the next Insight the countries that have seen the most significant increases in median age are these ones and again it gives an explanation as to why their median age might have risen more than any other for example Albania has seen significant emigration of younger people which could also lead to an older median age is it me or is that kind of crazy that it crunched all the data visualized it but then also gave really interesting analyzes of the data now you can read the other analyzes but each of them are really interesting and the final visualization which I asked for is brilliant I think notice how the graph goes from green to red when you get to the Future projection I didn't ask it to do that now obviously in this video I'm going to focus on the flashy visuals and the cool little tricks it can do but in terms of data analytics that is what is going to change jobs change Industries and remember this is code interpreter Alpha version one look at the difference between mid-journey version one and now mid Journey version 5 a year later how about basic video editing now there is a limit to what it can do but it can do some basic video editing if you ask it for example I uploaded a short file and asked it to rotate the file 180 degrees and it was able to do it now I'm not saying that is massively useful but it was able to do it here is a similar example I uploaded an image file and then said can you zoom out from the center of the image now initially it did zoom in but then I clarified that I wanted it to zoom out from the center just to be cheeky I also asked can you make it black and white oh and I also asked to add music but it couldn't add music anyway here is the end result by the way it gave it to me as an mp4 file and look it zooms out from the center and it's made the image black and white now because I got access so recently I honestly haven't explored the limits of what kind of video editing I can do with chat GPT code interpreter but I will let you know when I can now back to visualizations I gave it a hypothetical scenario that sounds kind of realistic I sent 231 CVS got 32 responses 12 phone interviews three follow-up face-to-face interviews and one job offer which I rejected I'll put a downloadable Sankey diagram of this data I did then get it to change the coloring slightly but I think that's a pretty cool Sankey diagram look sent CVS 231 and then receive responses and you can go down 32 phone interviews 12 face-to-face interviews and three job offers and one rejected offer obviously I could have tweaked that for hours make it more visual make it more interactive maybe make a gif of it but for two minutes work I think that's a pretty interesting and incredible output next and here is one that you might say is a little bit concerning and it's about steganography now I will admit I am not at all an expert in fact I know virtually nothing about it essentially what it involves though is hiding a message inside an image or in inside some code and gpt4 was more than willing to play along and it encoded a secret message into an image there was the image by the way and if you looked at that you'd think that's totally normal that's just a silly little image right well apparently here's what it can do to a casual Observer it looks like a simple image with some shapes but it actually contains the hidden message hello world then it provided a python function which can be used to decode the message from the image now obviously this is just a silly example that is totally harmless but am I being crazy in thinking this is a somewhat concerning ability for future language models to possess especially when they reach the level of an AGI often openai talk about future versions of GPT doing scientific research and finding things that humans wouldn't have discovered but let me pose the scenario that it gets better than any human expert at steganography anyway enough from me I'll let the experts weigh in on that one next did you know that gpt4 with code interpreter can do to text to speech just before anyone comments though why did I write proceed without further question because GPT 4 with code interpreter has a tendency to always ask clarifying questions and if you have access to only 25 messages every three hours you don't want to use up half or more of them on clarifying what it wants to do or saying yes please do that but I found writing proceed without further question means it gets straight to it and essentially you get double the number of prompts for your money anyway as you can see I asked turn this entire prompt starting from the beginning into a text speech file now quite a few times it denied it had the ability to do this but eventually I got it to work it was actually when I finally gave it this prompt and it worked I say it worked but it didn't quite work as intended check it out here is the text-to-speech that it came up with a large language model trained by open AI when you send a message containing python code to python it will be executed in a stateful device a notebook environment python will respond with the output of the execution or timeout after 120.0 seconds internet access for this session is disabled do not make external web requests or API calls as they will fail now thank you Stephen Hawking for that message the only thing is it had nothing to do with my original prompt now anyway when you get access to code interpreter play about with text-to-speech because it is able to do it even if it denies it time for a fun one I asked create a tree map of the letters in the following quote and I'm not going to read it out because I am not good at tongue twisters anyway I said give each part of the tree map a different color and output a downloadable file proceed without further question and here is the output and I checked it for the letter P and it was correct that there were 36 instances of the letter P in the output and look how it's proportional with the number of instances of the letter and the size of each rectangle I think that is pretty insane okay back to something more serious I uploaded this file which is an image of a math problem quite a hard one as well and you guessed it I said solve the math problem in this image it then extracted the text from the image presumably using OCR and then proceeded to solve it and I'm going to get onto this in a second it is better at math than Wolfram Alpha I know that's a big claim but it's far less buggy I found Wolfram Alpha crashing very frequently anyway here are the two solutions and isn't that incredible from a photo essentially it then extracts out the math problem including the two square roots and then solves it this is all within the same window of chapter no need for any other apps or extensions next it can do radial bar plots which I think are really quite beautiful I'm not saying this is the best one ever and I'm sure you could tweak it to make it more clear and beautiful look at that the life expectancy in the US climbing from 1800 and then it goes clockwise reaching a projected almost 90 by 2100 again I'm sure you could do a far better job than me in extracting out a more beautiful diagram but aren't radial bar plots just beautiful to look at speaking of cool diagrams how about this I didn't even specify which visualization to do I uploaded this same life expectancy data and I just said what are the most advanced and Technical visualizations you can do with this data proceed to do them now honestly it picks some visualizations that I don't think are the most advanced but nevertheless it was creative here is what it did it does frequently make the mistake of cluttering the axes and having far too many labels so that you can't see anything so scrub that one out not great but what about the next few remember it just did this on its own this is a heat map and you can see some really interesting things from this data like India starting with a much lower life expectancy than anyone else but gradually Rising but still falling behind the others even in 2100 and look at China look how the life expectancy drops in the 60s and 70s I think we all know what happened there compare that to the US which is a gradual continual Ascent actually aside from 22 20. look how the shade gets a little darker in 2020. obviously you guys can probably work out what happened around then but then the projections are for it to go up toward 90 by 2100 that's a beautiful and clear heat map that I didn't even ask for it to do let's look at the next one box plot do you remember those from school you get the upper end of the data the highest one the lowest one the median the first quartile and third quartile and it's a great way of statistically representing a set of data and it's done it for every 50th year starting in 1900. obviously a slightly less beautiful diagram than some of the ones you've seen today but for the statisticians in the audience you will know that this is a very useful metric for a lot of data the individual points above and below are typically when there are outliers in the data I would estimate that all of these visualizations only took around two two and a half minutes so definitely not the 10 seconds as I said that you often see on Twitter I mean have you ever seeing gpt4 give an answer in less than 10 seconds speaking of useful I think many professionals will find the next thing that I'm about to showcase the most useful of all any insights that Gypsy 4 finds Trends medians analyzes whatever you can ask it to add to the original file and then download it do you remember that the original file was called median age years well notice this file name median age years with insights it has created a downloadable new file with the insights included and look at some of the insights that I mean you have the change from 1950 to 2100 and here is the average median age throughout the period and the change from 2023 to 2100 notice that the original file didn't have those columns they were added by gpc4 with code interpreter and now how about data progression video files I was honestly shocked when I saw that it could do this but I asked can you make a 256 by 256 MP4 that gradually reveals the lines as they progress on the x-axis this was about the median age over time here is what it did and look at how the data and the chart progresses as time moves along I was really shocked to see this and the line in red which is going to be labeled at the end is the global median age and remember it calculated that that wasn't in the original file now I'm not sure why it picked out these four countries maybe because they represent extremes either way I think the result is phenomenal and I'm genuinely impressed that it did this even though I know the final result could be improved dramatically for example far higher resolution and maybe the global median age labeled from the start and actually now that it's got to the end I can see why it did pick out these countries because Niger did have the lowest median age in 2100 and it looks like Puerto Rico had the highest and the fastest aging one was Albania next and this this is going to shock quite a few people what about image editing I created this image in mid-journey version 5 and then here's what I asked I said use opencv to select the foreground of this image and look what it did it picked out the foreground no Blue Sky now I know it's not perfect but it's nevertheless impressive all within the window of chapter BT this does actually make me wonder if open Ai and chat to BT is eventually not now but in a few years gonna swallow all other apps or maybe Google's Gemini but either way one interface one website one app doing the job of all others and by the way of course chapter BT is now available on iOS but imagine you have one app and it can do image editing text-to-speech video editing everything data analysis not add gpt4 levels but GPT 6 or gbt 7 levels if you can get every piece of information service and application in one interface a bit like now people being addicted to their smartphones won't people be a addicted to this one interface again that's not going to happen now but I'm just posing it as a question to think over for the moment though before anyone gets too carried away it does still hallucinate quite a lot so I uploaded this image and I asked it questions about it and it answered and I was like wow it can do image recognition it said this image appears to be a digital painting of a humanoid figure at a desk with a rather complex background I was initially amazed until I realized that it probably got that from the file name because when I asked it questions it got it wrong so I said what is on the desk now look back there's this weird kind of microphone and a bit of paper and not much else a keyboard and look what it said there are multiple floating holographic displays okay a mouse not really a desk lamp I can't see that and then tools and devices now correct me if I'm wrong but I think most of those are incorrect now obviously I need to do far more experiments to see if it actually can recognize any particular images and maybe I'm putting it down too harshly but at the moment it does seem to hallucinate if you ask it about too much of the detail of an image next remember how one of the key weaknesses of GT4 is that it can't really count things especially not characters words Etc and even more so it can't do division and some of you might be thinking well with Wolfram Alpha it can do those things not quite here is an example of the code interpreter plugin essentially eating Wolfram Alpha obviating it making it not obvious what the utility of it is if you've got code interpreter I asked divide the number of the letter e's in this prompt by the number of the letter T's now you might think code interpreter can improve things by doing the character counting but it can also do the division notice how it counted the characters correctly compared to Wolfram Alpha and of course got the division correct as well so if it can do Advanced quadratics and do division and character counting Etc it does beg the question what would we use Wolfram Alpha for that we can't use code interpreter for I honestly might not know something that you guys know so do let me know in the comments it also also got this math question correct and notice you get these beautiful map visuals that you don't get with the base version of gpd4 you get something more like this where the visuals aren't as clear and notice the base version of GT4 gets the question wrong it can't do division but with code interpreter it gets the question right next one is a quick one pie charts nothing too special but I think it is a fairly beautiful visualization it doesn't seem to matter how big the CSV file is that you upload this next example was really quite fascinating it was a word puzzle I have tried this particular word puzzle on gpt4 dozens of times the reason I picked this puzzle is called a Word ladder is because it really struggles with the puzzle if the number of steps required is more than a certain number usually about five or six steps it gave me a really interesting border of the limits of gt4's planning abilities with language anyway it always gets it wrong here is a demonstration with the base model of gypsy 4. you might say why is this wrong but look at how it's changed change from Seas to sags which is more than one letter change and that's typical of the kind of Errors it makes what about with code interpreter well you can probably guess the ending given that I featured it in the video but it gets it right I believe it draws Upon A hard-coded word set and this does Point towards the kind of puzzles that I think gpc4 with code interpreter will be able to solve things like crosswords and sudokus okay not exactly world changing but nevertheless I think quite fascinating and how about Venn diagrams the reason I picked this example is that I had to go through about 10 steps to get it to create this rather basic three-way Venn diagram this represents the overlap between dogs Ai and desks and apparently all of them are loyal companions well we will see about that but anyway it took quite a few steps to get it right which was pretty annoying but here's the really interesting thing once I got it set up in the way that I like all I had to do was say use the format above to create a new three-way Venn diagram this time for Mango's Movie heroes and marmosets try to make each entry funny and use different colors proceed without further questions so it may have been a struggle to set up initially but once done it was so easy to iterate a new three-way Venn diagram and actually it was better than the original apparently all three are adored by fans worldwide apparently only marmosets and Movie heroes can climb up trees really fast and mangoes and marmosets can hang upside down that's crazy one or two prompts iterating on a design already agreed upon this is honestly what is likely to happen in the future with people spending hours to find the perfect data visualization or piece of data analysis and then just hitting copy paste for all their other files perfect it once and then it does the rest for you a quick couple of bonus ones before I finish you can just ask it to come up with a visualization giving it no direction at all it came up with a distribution of prime numbers up to ten thousand thing is I believe there's a slight mistake at the beginning because I think there's only 25 in the first 100 and 21 in the next 100. so you probably do want to still check the outputs that code interpreter gives you and that's another reason it's not going to instantly replace all data analysis and data visualization it's not perfect and it's not fully reliable but you've got to look ahead to where things are going I'm going to end where I started with this insane 3D surface map of a volcano if this is what gpd4 can do now with the Alpha version of code interpreter what will GPC 5 or 6 do with version 7 or 20 of code interpreter I was about to speculate about that but then I got distracted with trying to get inside this volcano it is kind of fun look I'm going above and into the volcano let me know what you will try when you get access I know they're rolling out steadily and I know that some people will have had access to it for about three weeks so hopefully if you want to experiment with it you will be able to soon in the meantime do let me know if you have any ideas that you want me to experiment with and thank you so much for watching all the way to the end have a wonderful day

Channel: AI Explained

Views: 528,339

Keywords:

Id: O8GUH0_htRM

Length: 24min 23sec (1463 seconds)

Published: Sat May 20 2023