The Truth About Google’s Gemini AI

all right so Google just came out with an announcement that is their answer to GPT 4 and GPT 3.5 in its current state and yay they're coming out with a model that is better than gp4 and this is both very exciting and interesting and very disappointing so in this video we're going to talk about both plus everything you need to know as a user of these products because a part of this announcement is available today for Google's B but more on that later because first things first we need to talk about the ginormous announcement here Gemini Ultra their most capable model moving forward so I'm going to give you a super concise summary my take on it and what I make of other people's commentary on this cuz I feel like a lot of it is displaced so first of all the summary it's multimodal from the ground up not as an afterthought as is the case with gp4 based on their design which of these would go faster the car on the right would be faster it is more aerodynamic that's amazing some of these use cases are mind-blowing we'll look at them in a second it actually beats gb4 on most benchmarks which is amazing this is a truly Next Level model that will be coming sometimes early next year so yeah the numbers check out here from what they're showing what they're telling us this is really impressive now Google Executives will be able to maintain pH in front of their board members and shareholders at the end of 2023 and users will be getting Next Level Tech in a few months I like it that's a win-win but here's the biggest deal okay so we just crossed a specific score on this MML U Benchmark and the score I'm talking about is this 89.8% on The mlu Benchmark this is what human experts achieve when taking that test okay so this is the very first model we will get access to that is actually better than human experts add specific tasks yikes so as you can see the Gemini Ultra scores 90% on this and gbd4 scores 87.2 n but they actually tried to fool us here n no not happening on this channel though because look in the blog post they're comparing the performance of Gemini Ultra and GPT 4 but they're not using the same prompting technique okay because what this means is that they prompted Gemini Ultra with 32 prompt prompts with a prompting technique called Chain of Thought and we're not going to get into that here but it's just a specific way to prompt to get the highest quality results and they did this 32 times picked the most common one and that's what this score represents okay so that's a fair technique to evaluate it right they used prompt engineering to get the best result possible but what they're comparing it to is to a five shot approach on gp4 which is a different and generally speaking less capable way of prompting the model and the funny thing is if you go into this paper which most shareholders most consumers don't you will find that if you compare the actual numbers it's 9.04 and 87.2 N so yes it's a difference yes it's Improvement but it's not the Improvement they make it out to be here just be aware of that but other than that it just beats gbd4 across the boards so it's fair to assume that this is a better model than gp4 so one more important fact here is that this model will have a 32k context window which does not match GPT 4 128k that we got Now doesn't work perfectly though we talked about that right it only works really well until somewhere around 64k token so this will be smaller than that but honestly 32k is plenty for most use cases so I'd rather have this model right and honestly I could go on for another 15 minutes here but I feel like those are the most important points for consumers but now let's talk about the multimodality of this cuz this is actually very different traditionally multimodal models are created by stitching together text only Vision only and audio only models in a suboptimal way at a secondary stage Gemini is multimodal from the ground up so if you're really into this I obviously recommend watching all of these we're not going to do that together here but they do a great job of visualizing this concept of how current models are built AKA taking a bit of a jab at gp4 because gp4 you might know that when it launched it was text only because it's a text only model and it was trained to be text first and then they added di vision and more and combined all of them into one interface where Now by just prompting something it can access those different models when you're a developer this is apparent because these are just different API endpoints that you can independently access but inside of cat GPT they easy to use web interface it combines all of them into one this is different with Gemini Ultra and they has 3% better performance in certain benchmarks all well and good I'm sure it's going to be fantastic once we get it but the fact that it's multimodal by Design that's actually a game changer and it just becomes really apparent in the use cases and before we look at those I just want to point out this also takes video which is huge right CH GPT doesn't it but this takes video inputs so yeah it's fair to say that from everything we see this is better than gp4 more capable so here's the one video that I would strongly recommend you watch watch fully show it to your friends show it to your family show this to everybody you deeply care about because this is literally what's going to be driving the future of software come up with a game idea oh and use emojis please how about this we can play a game called guess the country sure let's do it clue one this country is the home of the kangaroo the koala and the Great Barrier Reef oh that's easy clue right so it can do things like this and did you realize how fast that is how Snappy the latency on this is super in the demo and I believe this should be the final version right Google surely would just lie to us in a demo video right well that's what I thought yesterday but now that I'm finishing the edit of this video it turns out that some sloof on Twitter uncovered the fact that the prompts that they showed off in this use case video weren't the actual prompts the model was answering to so they did in fact doctor this demo to look better then the model is actually performing and that's a problem with these announcements they tell us something is coming in a few months show us fancy demos but they don't actually give us anything to test so we just got to blindly believe what they show us and it doesn't always turn out to be true so I don't think this is a major thing just something to be aware of right now let's get back to the video magic tricks analyzing cookies fidget Spinners and then even harder problems including reasoning and translation and look at that these are the video use cases what movie are they acting out here I think they are acting out the famous bullet time scene from The Matrix ooh nice so that's incredible right it takes in video it takes in the The Voice you could surely type and it's just interacting with the world so if you unleash this onto your desktop and you let it interact with your entire interface plus maybe you turn on the camera it could pick up your facial expression while picking up your voice while picking up your desktop while picking up the context that you provided with from other Google apps like your Google or YouTube search history and all the data they have on you that could be your personal context right so very quickly this will become insane so obviously they don't show some of the things that I just pointed out here but it doesn't take a lot of speculation to kind of just take the next step here right this will be the assistant of the future and they just demoed it to the world so yeah just make sure everybody you love sees this use case video so they understand where the world is heading so once we arrive there in a few months they won't be as surprised okay so that's the multimodality I should also note that several people on Twitter and myself tried some of these use cases with chat gp4 Vision across the boards they just don't work as well and I think they might have just picked out these examples so people do that so they arrive at that conclusion it's just that much better at the multimodal aspect of it okay but hey let me just briefly say we're comparing a 2024 model with a 2023 model with gp4 so it's not exactly Apples to Apples just wait for open AI to make their move with GPT 5 and then we will return to this comparison right but as of now Google is winning the announcement game whereas gp4 is winning the usable product game okay but that's not all there's a few more points that we really need to talk about here and one of them is the developer ecosystem and wait a second before you just skip this I think this is a really important point because one of the biggest things that openi has achieved over the last year since chat gpd's release they built that developer ecosystem that is beyond what any other AI provider like Google anthropic or the others can claim and honestly that might be the big differentiating factor what gets them ahead a lot of apps will be built up on gp4 just because it's already available and you can build stuff with it plus there's a strong argument to be made that you might just prefer a company that is solely focused on supporting developers and shipping AI models rather than Google that does 10,000 things and yes AI is one of their main priorities but just look at the landscape of their models and their AI technology there's a lot right you have the B chatbot you had the Paul models now you have the Gemini model and there's free Gemini models between the pro the Nano and the ultra then you have Google doet inside of their sheets then inside of Google Cloud they have a specific Ai and all of these seem to be led by independent teams with no really useful end point for developers to work with I don't know I just wanted to underline that the developer ecosystem that open a has it's just unmatched and I don't see that changing so fast even though this might be a few per better because again in this game of tennis usually open AI was making the matches and now Google got to make their move but as we know them there might be announcement lurking right around the corner as they usually do when a competitor comes up with something that trumps their models anyway getting back on topic there's two more really important points that we need to talk about here okay one is the other models they released and the other one is the new and upgraded Bart okay as mentioned they came out with free models Gemini Ultra their new flagship model AKA The Superior gp4 Gemini Pro which is essentially the 3.5 it's better at certain tasks worse at others it Nets out to be essentially the same so that's the pro model that's what's available in BS today and then Gemini Nano and the big deal here is that they actually released two versions of this one with 1.8 billion parameters and one with 3.25 billion now why is this a big deal we haven't really received models that are this compact yet right the most popular small model right now is Moll with 7 billion parameters but with 1.8 billion it will be really easy for developers to put this on phones okay so expect a lot of this technology to come to your smartphone in 2024 because we're getting models like the Nano anyway this also isn't available yet it will come to the pixel 8 next year but the thing that is available today is their pro model inside of bar today so how good is this I went into it I tested it a bunch I ran some of my standard prompts and here's my verdict it's a different flavor of GPT 3.5 is it better is it worse it really depends on the prompt and it's quite subjective now the cool thing is that they did add a bunch of extensions that could be useful to you but honestly in this date I have kind of a hard time recommending this over chat GPT to anybody the free version that is right they both just work and this is certainly better than the BS that we had last week for example because that was quite the mess to be honest and now they upgraded it to a level where it's decent it's good and I should point out that all the prompts that we discussed on this channel including the prompt factory work seamlessly in here so look everything you learned from this Channel and other prompt engineering resources you can still apply in here right all these large language models take natural language as an input and all of the techniques universally applies so if you learn how to craft a prompt once yeah it's just a universal skill but what is pretty cool are these prompt presets I'll say that because if you click them these prompts are actually proper prompts as opposed to what you get inside of cat GPT which is just very simplistic beginner prompts that I wouldn't really recommend at all the B prompt presets here are actually pretty good and some of them go sufficiently into depth to get proper results as we talked about so many times on this channel right by the way if you want to learn how to prompt like this and get weekly Live Events by me for the next 2 and a half months check out my prompt engineering course but yeah Bart with upgrade nothing crazy nothing to lose your breath over Gemini Ultra coming early next year yeah fantastic but let's be real we're also expecting GPT 5 early next year so they will have to compete with that and not with GPT 4 as they're comparing it with right now so if it was out today insane game-changing announcement what we got here with the pro model is okay they basically caught up so look what's the overarching story here well first of all finally Google made a move because they're considered by many as D tech company and they were getting beat by a little startup called open AI on arguably the most important front of their entire business which is AI including large language models consumer products but also developer endpoints and now they can finally claim that the tech giant is actually delivering and living up to the super high expectations that everybody involved in Google has and secondly it's the fact that we have so many players in this game now that it's just a question of time until we get commoditized models because you have Chinese companies with models like quen that are capable Beyond GPT 3.5 and the pro model that we got here today you have anthropic pushing the envelope now with a 200 200k context window this Ultra model that will get in 2 3 4 months maybe has a 32k context windows so it's not just better across the board right the leading in multimodal which is arguably the most important aspect of these models for user friendliness and use cases but then you have players like meta releasing translation models open source models like llama for everybody to build on freely that are out since months so who knows what they're cooking up so if it hasn't been obvious yet it's officially on the AI arms race is happening everybody's trying to get the future ahead of their competitors and the most aggressive and Innovative player open AI hasn't made their 2024 move yet so while seeing all this and especially seeing these use cases is extremely exciting I think it's fair to compare it to GPT 4 as a lot of social media is doing right now including this video but the only fair comparison is comparing this to GPT 5 that we essentially don't know anything about yet except of all the mathematical capabilities of qar and those leaks actually being confirmed by Sam Alman if that's a part of GPT 5 and it's fully multimodal well then it's going to be Google turn again to one of them or one of the 10 other companies competing on this so the only thing that's for sure is that 2024 is going to be as wild as this year and I'm here for it I'll be covering it all the use cases and what it means for you and on that note I would like to point out a new show that I'm starting this week which is a regular coverage of all the AI use cases that come out it's not a new show it's not covering all the latest drama in the AI space it's just covering the tools that you can put to work today that are actually worth covering so you can find the first episode over here it's coming out later this week all right so that's it check out this video it's a really good one

Channel: The AI Advantage

Views: 23,958

Keywords: theaiadvantage, aiadvantage, chatgpt, ai, chatbot, advantage, artificial intelligence, gpt-4, openai, ai advantage, igor

Id: gkykoHcqcbc

Length: 13min 29sec (809 seconds)

Published: Thu Dec 07 2023