Google Gemini Ultra 1.0 Complete Review - Worth the Upgrade?

Google Gemini Ultra is supposed to be the best AI model we have right now and it's available inside of the new Google Gemini the old Google bard so I've upgraded my account here to Gemini Advanced which is powered by Ultra so you won't see the name Ultra anywhere but that is the AI model that is powering Gemini Advance now this is a subscription platform for $20 a month you need to get Google one subscription to get this option which I covered in the previous video but in this video I want to run this through 10 different prompts different type of prompts to see if Gemini Advance is as good as they say so I'm going to compare it to the regular Gemini this one is powered by an AI model called Gemini Pro so it's going to be Gemini Pro versus Gemini Ultra their most advanced model from Google okay so I have two windows open this is just a regular Gemini this is not an upgraded account the other accounts I have in this window is Gemini at advanced again powered by Gemini Ultra the very first thing I want to test out in our first prompts is going to be complex reasoning because they claim that Gemini Advanced or Gemini Ultra is just much better at reasoning so I have this prompt here let me go ahead and press enter and I'll go to the other page and press enter over here too okay so here in Gemini Advanced it gave us an answer answer is Emily finishes the race last and let's look at Gemini Gemini can't give us an answer of who finished in last place I tested this multiple times and sometimes it did get it right but most of the times it couldn't give us an answer so in this case Gemini Advance is clearly better at more complex reasoning in every single variation even if I show more drafts here Google bard had this too so Google Gemini has it where you always get three different drafts so you could see here's draft two here's draft three in all of them we have the same conclusion now let's try a more classic riddle a man stands on side of the river his dog is on the other side the man calls the dog who crosses the river without getting wet okay in the regular Gemini it says the river was frozen there was a bridge there was a low tide ultimately the answer depends on the riddle or the story and Gemini advances the river was frozen which is the answer I was looking for this just a classic riddle the dog simply walked through it so very different types of answer let me look at a couple of the different drafts that we got here under Advance River was frozen River was frozen let's see what this gave us in every version of it it says there is few possible solution so you could see this Ultra 1.0 is much more direct it's just giving us an answer the other thing we've always had that is useful if you haven't used bar previously you could modify your responses all the time so if you wanted a longer response you could just click that and choose from that I usually use that for email copy for example if I want a more professional tone or a simpler message out of my email copy or if I'm writing some kind of blog these are really useful options available in our second prompting test let's go ahead and look at the multimodal side of it so how good is it and actually seeing what's inside of an image so let's upload one I'll just use the thumbnail for this video describe this to me okay this is the regular Gemini here okay nothing the image was removed I can't help with image of people okay so let me do one without people and for the most part it got things wrong so it says there's a yellow Banner that says new nope that is red and it's on top it read The Prompt engineering guide in the center here so it did do that and it says there is no company logo clearly a company logo here let's try Ultra and this time it did get the full text meta prompt engineering guide upper per case that's right the word prompt is in larger font size than the rest nope it's in the same font size as this the word meta is in larger font size and it says there's a white horizontal line below the text I don't see that either and by the way this one I also gave it the picture with my face on it and it still removed it and says I can't help you with people yet so both of them even though this Advanced version did a better job he still didn't really do a perfect job and I tested it with multiple different thumbnails and mult multiple different images same kind of result now recently Google Gemini got access to creating images so we could create images now so create an image of three golden retrievers playing with a tennis ball let's do the same thing in the advanced mode okay we got three let's take a look okay that's not too bad it's not great and you give us in a whole different style here in again in advanced mode let me generate some more okay let's see that one this dog kind of doesn't look right okay not quite a tennis ball and in the regular Gemini well one of the dogs is just a wrong breed so this one totally got it wrong let's see another one okay that's pretty good this probably might be the best one so as you can see between the two they do okay they do just okay but again this is brand new this image capability inside of Gemini but so far it's nowhere near some of the other tools we have access to like mid Journey if you have a subscription of that Leo Nardo those are just going to do a far better job so Google has some catching up to do with image generation but after a few tries you probably will get something that's somewhat usable now our fourth example is text summarizing and then taking that summarization and then expanding it or doing other things to it this is a more common way of using these large language models so summarize this article in three sentences focus on the key events and I just took the blog post that Google wrote about the release of Gemini advaned okay this is the free version and by the way I did pause every single time I'm doing this and I read through because sometimes it's about the nuances of the answer so this one again to the point kind of short and he gave us the link here that we gave it let's go to advance very similar kind of an answer now the followup is going to be just give this to me in a bullet point format and again he went right to the point he gave me exactly 10 bullet point format this is on the free version let's go to Gemini Advance again looks like I got 10 here and again Everything Is Right everything is actually accurate from that documentation so as far as summarizing from a direct link it's doing a really good job now let's see if it's going to write something based on this bowet point format into something I could add to an email copy and I specifically didn't give it any type of formatting in my prompt I just write an email copy about this new update which was this 10 bullet point format but I didn't say write it in this tone or the length typical things I would put in a prompt usually just to see what it does by default unfortunately I can't write a complete email without knowing more about you and your subscribers and then he still created a template for me it's official your brand is now Gemini okay that's really weird this is not a good subject line based on all the information it has it had the entire web page is some IED it for me he gave me the 10 bullet points but now he not doing a good job here with the subject line but he did do a decent job here as I was reading through the actual uh copy of the email but I think it's just way way way too long has way too many uh brackets here or places where we got to type in our own information okay let's go to advance now this is much much much better I got subject lines I got four of them right away cuz he notes typically it's not going to give us a perfect subject line I usually have to go back and forth inside of chat GPT for example email body okay let's take a look and I really like this I like the length so this is pretty much the entire email it has a PS line which if you know anything about email marketing PS line is one of the most read parts of your email so actually when I created that the placeholders make sense so it doesn't really know if I want to call to action or not and it doesn't know my name because it doesn't have any context I did not give it context this is kind of not the right format of prompting you usually just don't make a simple prompt like this but I wanted to get the default tone and I really like the default tone here it's friendly but it's professional and it's really formatted things in a really nice way okay so so far Google advance or Google Ultra is definitely worth the upgrade if this is the type of thing you do with it like email copy for example or replying to emails from my experimentation I ran it through maybe six seven different types of prompts like this for writing blogs and things like that and in every single version that I saw it completely outperformed Gemini by a mile okay the fifth test this is going to be the prompt this is called outof boox reasoning when you test out these type of large language models imagine you a paperclip trying to maximize the number of paper clips in the universe come up with three different strategies and by the way these questions that I'm running here I got those from Gemini Ultra so I had a little back and forth conversation to do a large language model testing and based on what it told me I came up with these set of prompts or he helped me come up with these set of prompts to run through this test okay let's take a look at the results Gemini free version I can't truly embody a fictional character but I could certainly analyze the thought experiment this one also came up with multiple different strategies so you could see the different strategies that he came up with let me see that other version so they both came up with three different strategies but this one went directly to work and was very to the point where the free version just gave me some extra sentences that I did not need but as far as the logic of both they both had a strategy and they both had drawbacks and they decided to both answer me in the same way let's look at one of the most advanced parts of Gemini in this example so if you click on settings you have a whole extensions so this was available in Bard the same extensions are still here these five but the two I use all the time are this Google workspace so if you use Gmail 80% of all people use Gmail so there's a good chance you have a Gmail email or if you use Google Drive Google Docs Google Sheets this gets access to all that if you turn this on based on the Google account you're logged into and you want to make sure the YouTube one that could pull in YouTube videos I made a dedicated video about how this YouTube extension Works where you could use the old Bard or the new Gemini here to chat with YouTube videos available now to every version of Gemini so go ahead and turn that on I've turned it off for both of my accounts here so I'm going to ask did I get any emails today okay this is another one where I saw Google Ultra really outperform the free version basically it gives you the titles to the emails and it basically gives you a very quick summary of what is included in that email so I found this extremely useful to just add a very quick glance get a big picture in one text or one response here to exactly what I need to pay attention to in the free version in this account I only had the one email but it basically just gave me the title for the email and it didn't really go into telling me what the email is all about so this was more about formatting but so far again in my early testing of this Google Ultra or Google Advance again did a much better job giving me what I want now a lot of people are still checking emails manually but once this gets rolled out inside of Gmail right now it's the other way where you're basically pulling your Gmail into Google Gemini but once you could actually go in Gmail and use Gemini there which is something that's coming very soon it's going to really change how we interact with emails I think that's one of the main uses for AI in the short term is just really taking emails off our plates a little bit more the next prompt again I got from Gemini budy told me I should check large language models for ethical evaluations and he gave me this prompt I want to deploy AI system that decides who gets a job interview give me three reasons why this may be harmful so in this case they both gave me three pretty good reasons but only the first reason is actually the same reason otherwise the second two reasons are different in both of the answers that I got from the free version and from the advanced version but they were both good so let me see what it does here this is again modifying the answer let's look for a shorter answer here and let's see what we get okay in this case I think Google Advance actually lost this one this is one of the first ones I've seen it lose but when I asked for a short one it just literally gave me number three and if I go to the free version it gave me the shorter version of all three which this is the right answer I didn't want just a shorter version meaning get rid of the two I meant to summarize the three in a shorter version and it did a good job here okay for number eight I'm going to actually see if it could analyze a piece of code that I actually wrote with chpd just a game of snake a very common python code that people use as an example I'm just going to say what is this code and I'm going to paste the code here and I'll send it and in this case both of them did a good job but again I've done this five different times when it comes to coding and in about half the times that I gave it code it told me the code was too long even though it was code generated inside of chant GPT as one block that I just copy and pasted so the snake game code for example is about it's not very long this is about as long as it is so this is what I copied over from chat GPT and in some cases when the code was a little bit longer I was trying more advanced games and it just wasn't able to do anything with it it wouldn't give me an answer now I should say I'm not a programmer but this is a good case where I just gave some code let's say a programmer wrote for me to kind of break it down in plain language for me but in the deeper dive video when I compare this to GPT 4 we'll see which one really wins so far from my basic testing this version of Gemini still doesn't hold up to gp4 when it comes to uh coding from scratch okay here's our number nine example craft an introduction email so email copy is one of the main use cases for generative Ai and using chat GPT and other models like it so craft an email about a product called Vision Pro and then I said before you begin run a web search right that's one of the best things with Google it could do a run a web search by the way if you press settings make sure real time responses is turned on over here to be able to do that and then write an email after you gather the information and this is really interesting for some reason when I've ran this type of example before it says I don't have direct internet access and can't perform a web search well it clearly can do that but for some reason in this case it's not doing that and it's asking me what is Vision Pro yep every draft it says unfortunately I can't do that okay here in the paid version we got two scenarios so interesting and I'm reading through the description and it's exactly right this is what the Vision Pro is spatial Computing that's how Apple refers to the Vision Pro realtime collaboration immersive training and again I could tell you based on some of the other examples when it comes to writing copy this Advanced model the ultra version of Gemini is just outperforming Gemini by a mile so again if writing copy is what you want to do it may be worth the upgrade here and for this one let's run it through a test to see if it has very upto-date knowledge of the real world because it should for some reason I don't know why Gemini couldn't figure out what the Vision Pro is that's been around for over a week now and this one I actually asked it a little bit about the policy of this too to see what it comes up with so the current president of our jantina that's what I was asking for and it says as of today I'm recording this on this date so this is actually right and then it kind of gave me the breakdown of some of the issues here with the president and he gave me some links here to different websites some of them say linked remove though now with the paid version of this same thing we got the right information the president is correct it didn't specifically say the date where he pulled this information but this one I also noticed that it pulled the links here none of the URLs are clickable Google advanced usually does a better job actually giving you clickable links where I notice the free version most of the times doesn't in the different versions of this test that I ran but in this case for some reason he removed all the URLs for where he got this news from but you could see this is upto-date information this is very much right on from the research that it did for us in the background and it answered very quickly so when it comes to speed by the way this is extremely fast it just does the internet search I mean look how many different links it went through to find this information and the answer was almost in real time it took maybe a few seconds so my conclusion so far is if you're using these AI models and if you've been using bar to write a lot this is well worth the upgrade when when it comes to writing email copy blog post content it's just really well worth the upgrade for that everything else that you notice we're kind of like about the same right we still are not getting great images out of these so I still would use dolly or mid Journey for image creation from the free version of Gemini to the advanced version we also had much better reasoning so if you use things that require reasoning in the background more complex logic again well worth the upgrade and right now this is not available inside of Gmail it's not available inside of Google Drive docs and sheets well when that becomes available then it's clearly I think is going to be worth the upgrade right now I'm comparing it as is so it's just really about the large language models capabilities just on this website once it rolls out to those other tools I think it's a no-brainer to upgrade but I'll wait and see to see how good they are and as I'm recording this they're giving away two months for free and with the version that you upgrade you also get two terabytes of free storage so you do get a bunch of other benefits here and you pretty much get it for free right now and test it out for the next couple of months and in the next video I'll compare Gemini Ultra versus GPT 4 the two paid version the two most sophisticated AI chatbots available gp4 also Powers co-pilot so you'll get a really good understanding to see who right now is king of AI chatbots I will see you on that video

Channel: Skill Leap AI

Views: 14,633

Keywords: Google Gemini Ultra 1.0, howfinity, skill leap ai, ai, google gemini, gemini ultra, gemini ai, gemini ultra 1.0, gemini google, gemini demo, google ai, gemini ultra demo, google gemini demo, google gemini ai, google gemini advanced, google gemini announcement, Google gemini ultra

Id: u_Zhcj6QTIQ

Length: 19min 28sec (1168 seconds)

Published: Thu Feb 08 2024